Chetan postgresql partitioning

EnterpriseDB, Postgres Plus and Dynatune are trademarks of EnterpriseDB Corporation. Other names may be trademarks of their respective owners. © 2011. All rights reserved. PostgreSQL Partitioning Using Inheritance and Check Constraints Presented by Chetan Suttraway November 22, 2011 Chetan.Suttraway @enterprisedb.com

© 2011 EnterpriseDB. All rights reserved. Partitioning
Range Partitioning
Specific range of data
timestamp
List Partitioning
Specific list of values
name IN ( 'PUNE', 'BENGALURU')

© 2011 EnterpriseDB. All rights reserved. Partitioning via Inheritance
Master Table
Empty
CREATE TABLE orders( id INT NOT NULL, address TEXT NOT NULL, order_date TIMESTAMP NOT NULL );
Child Tables
Use non-overlapping Constraints
CREATE TABLE orders_part_2011 ( CHECK ( order_date >= DATE '2011-01-01' AND order_date < DATE '2012-01-01' ) ) INHERITS (orders) TABLESPACE tbsp2; CREATE TABLE orders_part_2010 ( CHECK ( order_date < DATE '2011-01-01' ) ) INHERITS (orders); No Primary Key! Match datatype on constraints Simple partition names Plan reading becomes easy

© 2011 EnterpriseDB. All rights reserved. Partitioning via Inheritance
Create Indexes on Child Tables
On key columns for each partition
CREATE INDEX orders_part_2011_idx ON orders_part_2011(order_date); CREATE INDEX orders_part_2010_idx ON orders_part_2010(order_date);
Constraint Exclusion configuration parameter
postgresql.conf
CONSTRAINT_EXCLUSION = ON;
Session
SET constraint_exclusion = 'PARTITION';
Database
ALTER DATABASE somedb SET constraint_exclusion = on;

Partitioning via Inheritance © 2011 EnterpriseDB. All rights reserved.
Trigger or Rule
Redirect data inserted into master table to appropriate child table.
CREATE OR REPLACE FUNCTION orders_insert1() RETURNS TRIGGER AS $$ DECLARE vsql Text; BEGIN vsql := 'INSERT INTO orders_part_'|| to_char(NEW.order_date, 'YYYY' )|| ' VALUES ('||NEW.id||','||quote_litera(NEW.address)||','||quote_literal(NEW.order_date)||')'; RETURN NULL; END; $$ LANGUAGE plpgsql; Low Maintenance Quoting, NULL issues Fast, No locking for updation of function
Either of Rules or triggers can be used.
Compute partition name on the fly.
Different variations for insert, update, delete.
CREATE TRIGGER orders_insert_trigger BEFORE INSERT ON orders FOR EACH ROW EXECUTE PROCEDURE orders_insert();

Partitioning via Inheritance © 2011 EnterpriseDB. All rights reserved. CREATE OR REPLACE FUNCTION orders_insert() RETURNS TRIGGER AS $$ BEGIN IF (NEW.order_date >= DATE '2011-01-01' AND NEW.order_date < DATE '2012-01-01') THEN INSERT INTO orders_part_2011 VALUES (NEW.*); ELSIF (NEW.order_date < DATE '2011-01-01') THEN INSERT INTO orders_part_2010 VALUES (NEW.*); ELSE RAISE EXCEPTION 'Date out of range. check orders_insert() function!'; END IF; RETURN NULL; END; $$ LANGUAGE plpgsql; CREATE TRIGGER orders_insert_trigger BEFORE INSERT ON orders FOR EACH ROW EXECUTE PROCEDURE orders_insert(); High Maintenance No quoting, NULL issues Fast, No locking for updation of function

© 2011 EnterpriseDB. All rights reserved. Partitioning via Inheritance pg=# INSERT INTO orders VALUES(1, 'pune', '2011-08-22'); INSERT 0 0 SELECT * FROM orders; id | address | order_date ----+--------------+--------------------------- 1 | pune | 22-AUG-11 00:00:00 2 | bengaluru | 22-FEB-10 00:00:00 (2 rows) pg=# INSERT INTO orders VALUES(2, 'pune', '2010-02-22'); INSERT 0 0 pg=# UPDATE orders SET address = 'bengaluru' WHERE id = 2; UPDATE 1 SELECT * FROM orders_part_2011; id | address | order_date ----+---------+------------------------ 1 | pune | 22-AUG-11 00:00:00 (1 row) SELECT * FROM orders_part_2010; id | address | order_date ----+-----------+-------------------- 2 | bengaluru | 22-FEB-10 00:00:00 (1 row) SELECT * FROM ONLY orders; id | address | order_date ----+---------+------------ (0 rows)

© 2011 EnterpriseDB. All rights reserved. Partitioning via Inheritance EXPLAIN SELECT * FROM orders WHERE order_date = '02-JAN-11'; QUERY PLAN ---------------------------------------------------------------------------------------- Result (cost=0.00..26.01 rows=7 width=40) -> Append (cost=0.00..26.01 rows=7 width=40) -> Seq Scan on orders (cost=0.00..23.75 rows=6 width=44) Filter: (order_date = '02-JAN-11 00:00:00'::timestamp without time zone) -> Seq Scan on orders_part_2011 orders (cost=0.00..2.26 rows=1 width=18) Filter: (order_date = '02-JAN-11 00:00:00'::timestamp without time zone) (6 rows) EXPLAIN SELECT * FROM orders WHERE order_date = '02-JAN-10'; QUERY PLAN ---------------------------------------------------------------------------------------- Result (cost=0.00..24.76 rows=7 width=44) -> Append (cost=0.00..24.76 rows=7 width=44) -> Seq Scan on orders (cost=0.00..23.75 rows=6 width=44) Filter: (order_date = '02-JAN-10 00:00:00'::timestamp without time zone) -> Seq Scan on orders_part_2010 orders (cost=0.00..1.01 rows=1 width=44) Filter: (order_date = '02-JAN-10 00:00:00'::timestamp without time zone) (6 rows

© 2011 EnterpriseDB. All rights reserved. Query Planning EXPLAIN SELECT * FROM orders WHERE order_date = now(); QUERY PLAN ------------------------------------------------------------------------------------ Result (cost=0.00..30.03 rows=8 width=41) -> Append (cost=0.00..30.03 rows=8 width=41) -> Seq Scan on orders (cost=0.00..26.50 rows=6 width=44) Filter: (order_date = now()) -> Seq Scan on orders_part_2011 orders (cost=0.00..2.51 rows=1 width=18) Filter: (order_date = now()) -> Seq Scan on orders_part_2010 orders (cost=0.00..1.01 rows=1 width=44) Filter: (order_date = now()) (8 rows)
Planner analyses the query before values from parameters or stored procedures are substituted.

© 2011 EnterpriseDB. All rights reserved. Constraint Exclusion
Works for range or equality check constraints.
No automatic way to verify that all of the CHECK constraints are
mutually exclusive
ALTER TABLE product_items_j ADD CONSTRAINT chk_item_name CHECK ( item_name LIKE 'J% ');
WHERE condition has to be similar to the constraint.
Constraint Exclusion works when WHERE clause has constants.
ALTER TABLE product_items_j ADD CONSTRAINT chk_item_name CHECK ( item_name BETWEEN 'J' AND 'JZ' ');
SELECT item_name FROM product_items WHERE item_name LIKE 'K%' ;
SELECT item_name FROM product_items WHERE item_name = 'Kettle ';
Always look at the plan!

© 2011 EnterpriseDB. All rights reserved. Inheritance
Child Tables inherits:
All NOT NULL and CHECK constraints
Column default values
Child Tables does not inherit:
Indexes
All other constraints – unique, primary and foreign key
Ownership
Permissions
Changes to parent are propagated to child
Cannot rename inherited columns on child tables
Can remove NOT NULL and default value on child tables
ALTER TABLE can enable or disable inheritance on child tables
Additional columns can be added to child tables.
ANALYZE all child tables individually

© 2011 EnterpriseDB. All rights reserved. Uniqueness
Primary Key and Unique Constraints
Index for uniqueness
No Multi-Table Indexes
Indexing Partition Keys- Columns
Non-overlapping check constraints
Unique Index over each Partition
Indexing Non-Partition Keys- Columns
Unique Index over each Partition
Custom functions to scan over all Partitions
- Not what we want!
Foreign Keys
Additional table with primary keys from both parent and child
Triggers to maintain this table

© 2011 EnterpriseDB. All rights reserved. Automating Maintenance
Pre-create deterministic partitions
Rules, Triggers, Functions can be dynamically created
Partition structure doesn't need to be complete
http://wiki.postgresql.org/wiki/Table_partitioning

© 2011 EnterpriseDB. All rights reserved. Querying Over Partitions...MAX pg=# EXPLAIN SELECT MAX(order_date) FROM orders; QUERY PLAN -------------------------------------------------------------------------------- Result (cost=26.56..26.57 rows=1 width=0) InitPlan 1 (returns $0) -> Limit (cost=26.52..26.56 rows=1 width=8) -> Merge Append (cost=26.52..73.51 rows=1196 width=8) Sort Key: public.orders.order_date -> Sort (cost=26.47..29.21 rows=1094 width=8) Sort Key: public.orders.order_date -> Seq Scan on orders (cost=0.00..21.00 rows=1094 width=8) Filter: (order_date IS NOT NULL) -> Index Scan Backward using orders_part_2011_idx on orders_pa rt_2011 orders (cost=0.00..14.02 rows=101 width=8) Index Cond: (order_date IS NOT NULL) -> Index Scan Backward using orders_part_old_idx on orders_par t_old orders (cost=0.00..8.27 rows=1 width=8) Index Cond: (order_date IS NOT NULL) (13 rows)

Chetan postgresql partitioning

by OpenSourceIndia on Dec 15, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Chetan postgresql partitioning — Presentation Transcript