MySQL Partitioning 5.6

MySQL Partitioning Internals
Mattias Jonsson
Senior Software Engineer

Program Agenda
§  What is partitioning
§  How partitioning is handled inside the server
§  New features in 5.6

§  Split a table into pieces
§  Specify which rows goes
where
§  Transparent
–  Still looks like a table
Horizontal Partitioning
What Is Partitioning

§  Easier to manage
–  Drop/exchange partition
§  Performance
–  Partition pruning
–  Smaller active indexes
Reasons
Why Partition?

§  Manually specify the range
for each partition
§  Can be subpartitioned
–  HASH/KEY
§  RANGE COLUMNS
–  Use native column type
–  More than one column
Range
Partitioning Types

NULL Handling
§  NULL is compared as less than any number
–  Always in the first partition in a range partitioned table

§  Manually specify a list of
values for each partition
§  Can be subpartitioned
–  HASH/KEY
§  LIST COLUMNS
–  Use native column type
–  More than one column
List
Partitioning Types

§  Modulus based
§  INT columns only
§  LINEAR option
Hash
Partitioning Types

§  Modulus based
§  All columns types
§  Multiple columns
§  LINEAR option
Key
Partitioning Types

Partitioning in MySQL
§  Parser
§  Optimizer
–  Pruning via range optimizer
§  Runtime
–  ALTER PARTITION
§  Engine
–  Generic partitioning engine: ha_partition

Partitioning Engine
§  Main engine of the table
§  Virtual engine, stores no data
§  Each partition is a handle to the real engine/data
–  InnoDB, MyISAM, Archive...
§  ”Proxies” Storage Engine API calls to its partitions
–  Read/Write/Update etc is forwarded

Some Limitations
§  All partition columns must be included in all unique keys
–  Including the PRIMARY KEY!
–  The index is partitioned with the data
–  The same unique key must be in the same partition
–  Enforced by the partitions storage engine.
§  Only a limited set of deterministic functions supported
–  Mostly used TO_DAYS, TO_SECONDS or UNIX_TIMESTAMP

PARTITION BY RANGE (part_col)
(PARTITION p1 VALUES LESS THAN (100),
PARTITION p2 VALUES LESS THAN (1000)
…
EXPLAIN PARTITIONS
SELECT * FROM part_t
WHERE part_col
BETWEEN 100 AND 999
Part Id
1 0
10
88
99
2 100
311
3 2000
Partition Pruning

PARTITION BY RANGE (part_col)
PARTITION p2 VALUES LESS THAN (1000)
…
UPDATE part_t SET Id = 212
WHERE Id = 88
Deleted from Partition 1, Inserted into
Partition 2
Part Id
1 0
10
88
99
2 100
212
311
3 2000
UPDATE

SELECT * FROM part_t WHERE cond
Reads from one partition at a time,
no extra buffers needed
Part Id
1 0
10
88
99
2 100
311
3 2000
Unordered read

ORDER BY index_col
Reads first matching row from each
partition
Part Index
1 9
10
12
22
2 8
12
3 13
Ordered read

ORDER BY index_col
Sorts the rows through a priority queue
and return the first row
Part Index
1 9
10
12
22
2 8
12
3 13
Ordered read

ORDER BY index_col
Refill the priority queue from that partition
Part Index
1 9
10
12
22
2 8
12
3 13
Ordered read

Optimized handler calls
§  Auto increment handling
–  Caching the max auto_inc value
§  Static calls
–  Use the first partition only
§  Index statistics
–  Use only the largest used partitions
§  Bulk insert

DDL Logging
§  All alter operations of partitions is logged
§  If Fail
–  Undo partition operations
§  If Crash
–  Undo operations if not committed
–  Complete operation otherwise
ALTER PARTITION is atomic

INFORMATION_SCHEMA.PARTITIONS
§  Partition level information
–  Partition name, subpartition name, partition expression, partition value/
description etc
–  Plus the same info as in I_S.TABLES per partition
§  Number of rows
§  Data/Index lenght
§  Etc.

Partitioning Operations
§  Table maintenace per partition
–  ANALYZE
–  CHECK
–  OPTIMIZE
–  REPAIR
–  REBUILD
–  TRUNCATE
ALTER TABLE t CMD PARTITION

Add Range Partition
1 100 199
2 200 299
3 300 399
ALTER TABLE t ADD PARTITION
(PARTITION p4
VALUES LESS THAN (500))
1 100 199
2 200 299
3 300 399
4 400 499

Drop Range Partition
ALTER TABLE t DROP PARTITION p2
1 100 199
2 200 299
3 300 399
1 100 199
2 200 299
3 200 399
4 400 499
1 100 199
2 200 299
3 300 399
4 400 499

Add List Partition
1 1,2,9
2 3,6,8
3 4,11,NULL
ALTER TABLE t ADD PARTITION
(PARTITION p4
VALUES IN (5, 10 ,15)
1 1,2,9
2 3,6,8
3 4,11,NULL
4 5,10,15
1 1,2,9
2 3,6,8
3 4,11,NULL

Drop List Partition
1 1,2,9
2 3,6,8
3 4,11
ALTER TABLE t DROP PARTITION p2
1 1,2,9
2 3,6,8
3 4,11,NULL
4 5,10,15
1 1,2,9
2 3,6,8
3 4,11,NULL
4 5,10,15

Add/Drop Hash/Key Partition
§  Rows will be redistributed!
ALTER TABLE t ADD PARTITION PARTITIONS 1
ALTER TABLE t COALESCE PARTITION 1
1 1,2,9
2 3,6,8
3 4,11
1 0, 4, 8
2 1, 5
3 2, 6
4 3, 7
1 0, 3, 6
2 1, 4, 7
3 2, 5, 8

Add/Drop LINEAR Hash/Key Partition
§  Partitions will only be splitted / merged
§  Even distribution only for 2^n partitions
ALTER TABLE t ADD PARTITION PARTITIONS 1
ALTER TABLE t COALESCE PARTITION 1
p0 p2 p1
0, 4, 8 2, 6, 10 1, 3, 5, 7, 9
p0 p2 p1 p3
0, 4, 8 2, 6, 10 1, 5, 9 3, 7

Reorganize Partition
1 100 199
2 200 299
3 300 399
ALTER TABLE t REORGANIZE PARTITION p2, p3
PARTITION p4 VALUES LESS THAN (500),
PARTITION pMax VALUES LESS THAN MAXVALUE)
1 100 199
23 200 399
4 400 499
Max 500 ∞

Removing Partitioning
§  ALTER TABLE t REMOVE PARTITIONING

Time of 50k single row
PK operations
Lock call overhead per
partition
Lock handling limits performance in 5.5
0
10
20
30
40
50
60
70
0 1 8 32 64 128 256 512 1024
UPDATE DELETE INSERT SELECT

Query processing in 5.5
Index read
Table scan
Condition evaluation
Sorting, grouping
Sending data to client
Prepare internal structures
Transformations
Optimizations
Partition Pruning
Query plan creation
Const table evaluation
Parse Open/Lock
Open all tables
Derived table
handling
Lock all tables
Prepare/Optimize Execute
Really pre-5.6.6

Query processing in 5.6
Index read
Table scan
Condition evaluation
Sorting, grouping
Sending data to client
Prepare internal
structures
Transformations
Partition Pruning
Parse Open
Lock all
tables
Lock Execute
From 5.6.6
Prepare Optimize
Transformations
Optimizations
Partition Pruning
Query plan creation
Const table evaluation
Open all
tables
Derived
table
handling

Time of 50k single row
PK operations
No more overhead per
partition!
5.6 Prunes Partition Lock Calls
0
10
20
30
40
50
60
70
0 1 8 32 64 128 256 512 1024
UPDATE - 5.6 DELETE - 5.6 INSERT - 5.6 SELECT - 5.6

Sysbench with 1 and 64
threads
Partitioning can increase
concurrancy!
Sysbench 5.5 vs 5.6
0
500
1000
1500
2000
2500
3000
1 16 64 128 256 512 1024
1 - 5.5 1 - 5.6 64 - 5.5 64 - 5.6

Sysbench with 1 and 64
threads
Now you can have 1
partition per day for > 20
years!
Now 8K partitions!
0
500
1000
1500
2000
2500
3000
1 16 64 128 256 512 1024 2048 4096 8192
1 - 5.5 1 - 5.6 64 - 5.5 64 - 5.6

Partitions in the Table Cache
§  At table open/close – All partitions are open/closed
–  Takes up resources in the table cache

Partition Selection
§  SELECT * FROM t PARTITION (p1,p4,p8) WHERE cond
§  UPDATE/DELETE/INSERT/REPLACE/LOAD
§  Also for subpartitions
Explicitly choose partitions used

Exchange Partition
§  ALTER TABLE t_part
EXCHANGE PARTITION p2
WITH TABLE t_non_part
Switch data between partition and table Part
p1 0
10
88
99
p2 100
311
p3 2000
212
214

Exchange Partition
§  ALTER TABLE t_part
EXCHANGE PARTITION p2
WITH TABLE t_non_part
Switch data between partition and table Part
p1 0
10
88
99
p2 212
214
p3 2000
100
311

Performance Schema
§  To avoid performance issues / exessive resource usage
–  Only table level is recorded

More Partitioning
§  Forum - http://forums.mysql.com/list.php?106
§  Doc - http://dev.mysql.com/doc/refman/5.6/en/partitioning.html

The preceding is intended to outline our general product direction. It is intended
for information purposes only, and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or functionality, and should
not be relied upon in making purchasing decisions. The development, release,
and timing of any features or functionality described for Oracle s products
remains at the sole discretion of Oracle.

MySQL Partitioning 5.6

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to MySQL Partitioning 5.6

Similar to MySQL Partitioning 5.6 (20)

More from Mark Swarbrick

More from Mark Swarbrick (20)

Recently uploaded

Recently uploaded (20)

MySQL Partitioning 5.6