Your SlideShare is downloading. ×
0
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Postgres rules
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Postgres rules

1,834

Published on

A brief introduction to Postgres Rules

A brief introduction to Postgres Rules

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,834
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • This is a presentation about the Rules feature of Postgres.\n\nIn brief, Rules are a sometimes more efficient alternative to triggers, which function by rewriting incoming SQL.\n\nThey can be massively faster than triggers, and they give you great flexibility in creating writable views, but the expectation that they should work like triggers when they don’t can induce some subtle and dangerous errors.\n
  • To speak of code in the server to a Rails using audience is of course to engage in heresy. Nevertheless, I believe that for performance, reliability and security reasons, code in the database can be just the thing.\n
  • Here is how we create a Rule\n
  • I should emphasize that Rules are esoteric. Meaning that for most purposes, you don’t need Rules. If triggers will work for you and are fast enough, use triggers.\n\nRules are for two specific uses:\n1. Writable views; and\n2. Code that runs on insert, update or delete that runs fast when you are doing a *bulk* insert, update or delete.\n
  • Why not just use a trigger?\n\nIf you do a bulk insert, update or delete, your trigger will run at least once for each affected row. This can be deadly slow. Many developers avoid triggers for this specific reason.\n
  • Worth pointing out that you’re probably already employing Rules: Postgres implements views using Rules.\n
  • A Postgres view is just a dummy table with a Rule that runs on SELECT that instead runs the query underlying the view\n
  • So speed is one of the reason to use Rules. A rule will only run once, no matter how man rows you’re affecting, so a Rule won’t be affected at all by the problem affecting triggers.\n
  • If you’re contemplating using Rules, please head a serious warning. You can really do things wrong if you don’t really understand how Rules work, or if you don’t test carefully.\n
  • Here is the page you have to read, *carefully* if you’re contemplating using Rules.\n\nIt’s dry as dust. I’ve encountered more readable EULAs. But there is information in there you *must* understand. Because your intuitions *will* grievously mislead you otherwise.\n
  • Here, for example, is the crucial point about an INSTEAD rule.\n\nYou *must* remember that a Rule doesn’t run its condition and *then* rewrite the query. Instead, it modifies the query and runs that, to similar but by no means identical effect.\n
  • One of the first things many ask about when they discover Rules is how to do an upsert (insert if not present; update if it is).\n\nSo they do something like this…\n
  • One of the first things many ask about when they discover Rules is how to do an upsert (insert if not present; update if it is).\n\nSo they do something like this…\n
  • One of the first things many ask about when they discover Rules is how to do an upsert (insert if not present; update if it is).\n\nSo they do something like this…\n
  • If you really stare at the conversion, you can see why the rule turns an insert of (1, 1) into a result of (1, 2). But it sure isn’t obvious from the Rule that this is what will happen.\n
  • If you really stare at the conversion, you can see why the rule turns an insert of (1, 1) into a result of (1, 2). But it sure isn’t obvious from the Rule that this is what will happen.\n
  • If you really stare at the conversion, you can see why the rule turns an insert of (1, 1) into a result of (1, 2). But it sure isn’t obvious from the Rule that this is what will happen.\n
  • If you really stare at the conversion, you can see why the rule turns an insert of (1, 1) into a result of (1, 2). But it sure isn’t obvious from the Rule that this is what will happen.\n
  • If you really stare at the conversion, you can see why the rule turns an insert of (1, 1) into a result of (1, 2). But it sure isn’t obvious from the Rule that this is what will happen.\n
  • Here is another example. Because the insert has a dynamic function in it, the query rewrite gives you different results in the log table than what was inserted into the original table.\n
  • Here is another example. Because the insert has a dynamic function in it, the query rewrite gives you different results in the log table than what was inserted into the original table.\n
  • Here is another example. Because the insert has a dynamic function in it, the query rewrite gives you different results in the log table than what was inserted into the original table.\n
  • Here is another example. Because the insert has a dynamic function in it, the query rewrite gives you different results in the log table than what was inserted into the original table.\n
  • Here is another example. Because the insert has a dynamic function in it, the query rewrite gives you different results in the log table than what was inserted into the original table.\n
  • Here is another example. Because the insert has a dynamic function in it, the query rewrite gives you different results in the log table than what was inserted into the original table.\n
  • Let’s see a real-world example.\n
  • I’ve a set of financial operations (credit card and cash receipts). These are “covered” by reconciliations. All the operations between a start and end time stamp by a particular operator are covered by the single reconciliation.\n\nThe question becomes how to maintain the relationship? We could include the start and end timestamps in the reconciliations table, but then to determine the reconciliation for a financial operation is a bit of a hassle.\n\nOr we could point a financial operation at its reconciliation, but we need to enforce that the financial operations in a block are all covered by the same reconciliation.\n
  • We put the foreign key on financial_operations because that’s reasonably easy to work with from either end of the relationship.\n\nThen we create a reconciliations view that pulls in the results of that relationship.\n
  • We put the foreign key on financial_operations because that’s reasonably easy to work with from either end of the relationship.\n\nThen we create a reconciliations view that pulls in the results of that relationship.\n
  • We create a rule in two parts:\n\nThe first updates the reconciliations_data table.\n\nNote the “returning” clause can go just fine on the first query. You can only have one “returning”, but it doesn’t have to be on the last statement.\n
  • The second part updates the matching financial_operations records.\n\nThe results are simple and effective, and we can update many thousands of financial_operations records with a single insert to the view.\n
  • \n
  • Transcript

    • 1. Postgres Rules
    • 2. Heresy
    • 3. CREATE RULE
    • 4. Esoteric
    • 5. Y u no use trigger?
    • 6. View
    • 7. ViewCREATE TABLE users ( id integer, name varchar(40), PRIMARY KEY(did));CREATE VIEW myview AS SELECT * FROM users;
    • 8. ViewCREATE TABLE users ( id integer, name varchar(40), PRIMARY KEY(did));CREATE VIEW myview AS SELECT * FROM users; ==CREATE RULE "_RETURN" AS ON SELECT TO myview DO INSTEAD SELECT * FROM mytab;
    • 9. Speed
    • 10. Warning
    • 11. Warning
    • 12. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification added
    • 13. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification added
    • 14. CREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL);
    • 15. CREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL);CREATE RULE upsert_counts AS ON INSERT TO counts WHERE exists ( SELECT * FROM counts WHERE id = NEW.id ) DO INSTEAD UPDATE counts SET count = count + 1 WHERE id = NEW.id;
    • 16. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification added CREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL ); CREATE RULE upsert_counts AS ON INSERT TO counts WHERE exists ( SELECT * FROM counts WHERE id = NEW.id ) DO INSTEAD UPDATE counts SET count = count + 1 WHERE id = NEW.id;
    • 17. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification addedCREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL);CREATE RULE upsert_counts AS ON INSERT TO counts WHERE exists ( SELECT * FROM counts WHERE id = NEW.id ) DO INSTEAD UPDATE counts SET count = count + 1 WHERE id = NEW.id;
    • 18. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification addedCREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL);CREATE RULE upsert_counts AS ON INSERT TO counts WHERE exists ( SELECT * FROM counts WHERE id = NEW.id ) DO INSTEAD UPDATE counts SET count = count + 1 WHERE id = NEW.id;INSERT INTO counts (id, count) VALUES (1, 1);
    • 19. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification addedCREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL);CREATE RULE upsert_counts AS ON INSERT TO counts WHERE exists ( SELECT * FROM counts WHERE id = NEW.id ) DO INSTEAD UPDATE counts SET count = count + 1 WHERE id = NEW.id;INSERT INTO counts (id, count) VALUES (1, 1); # SELECT * FROM counts; id | count ----+---------- 1 | 2 (1 row)
    • 20. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification addedCREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL);CREATE RULE upsert_counts AS ON INSERT TO counts WHERE exists ( SELECT * FROM counts WHERE id = NEW.id ) DO INSTEAD UPDATE counts SET count = count + 1 WHERE id = NEW.id;INSERT INTO counts (id, count) VALUES (1, 1); # SELECT * FROM counts; id | count ----+---------- 1 | 2 (1 row)
    • 21. Qualification given and INSTEADthe query tree from the rule action with the rule qualification and the original query treesqualification; and the original query tree with the negated rule qualification addedCREATE TABLE counts ( id INT4 PRIMARY KEY, count INT4 NOT NULL);CREATE RULE upsert_counts AS ON INSERT TO counts WHERE exists ( SELECT * FROM counts WHERE id = NEW.id ) DO INSTEAD UPDATE counts SET count = count + 1 WHERE id = NEW.id;INSERT INTO counts (id, count) VALUES (1, 1); # SELECT * FROM counts; id | countBECOMES: ----+----------INSERT INTO test (id, some_val) 1 | 2 SELECT 1, 1 WHERE NOT ( (1 row) EXISTS ( SELECT * FROM test WHERE id = 1) );UPDATE testSET some_val = some_val + 1WHERE id = 1 AND ( EXISTS ( SELECT * FROM test WHERE id = 1 ) );
    • 22. CREATE TABLE test ( id! ! INT4 PRIMARY KEY, val! INT4 NOT NULL);
    • 23. CREATE TABLE test ( id! ! INT4 PRIMARY KEY, val! INT4 NOT NULL);CREATE TABLE test_log ( id !! ! INT4 PRIMARY KEY, creation_date! TIMESTAMP NOT NULL, val ! ! ! ! INT4 NOT NULL,! CONSTRAINT test_log_pk PRIMARY KEY(id, creation_date));
    • 24. CREATE TABLE test ( id! ! INT4 PRIMARY KEY, val! INT4 NOT NULL);CREATE TABLE test_log ( id !! ! INT4 PRIMARY KEY, creation_date! TIMESTAMP NOT NULL, val ! ! ! ! INT4 NOT NULL,! CONSTRAINT test_log_pk PRIMARY KEY(id, creation_date));CREATE RULE test_logging AS ON INSERT TO test! DO ALSO INSERT INTO test_log(id, creation_date, val) VALUES(NEW.id, NOW(), NEW.val);
    • 25. CREATE TABLE test ( id! ! INT4 PRIMARY KEY, val! INT4 NOT NULL);CREATE TABLE test_log ( id !! ! INT4 PRIMARY KEY, creation_date! TIMESTAMP NOT NULL, val ! ! ! ! INT4 NOT NULL,! CONSTRAINT test_log_pk PRIMARY KEY(id, creation_date));CREATE RULE test_logging AS ON INSERT TO test! DO ALSO INSERT INTO test_log(id, creation_date, val) VALUES(NEW.id, NOW(), NEW.val);INSERT INTO test(id, count) VALUES (1, RANDOM());
    • 26. CREATE TABLE test ( id! ! INT4 PRIMARY KEY, val! INT4 NOT NULL);CREATE TABLE test_log ( id !! ! INT4 PRIMARY KEY, creation_date! TIMESTAMP NOT NULL, val ! ! ! ! INT4 NOT NULL,! CONSTRAINT test_log_pk PRIMARY KEY(id, creation_date));CREATE RULE test_logging AS ON INSERT TO test! DO ALSO INSERT INTO test_log(id, creation_date, val) VALUES(NEW.id, NOW(), NEW.val);INSERT INTO test(id, count) VALUES (1, RANDOM());# SELECT * FROM test; id | val----+------ 1 | 46228(1 row)
    • 27. CREATE TABLE test ( id! ! INT4 PRIMARY KEY, val! INT4 NOT NULL);CREATE TABLE test_log ( id !! ! INT4 PRIMARY KEY, creation_date! TIMESTAMP NOT NULL, val ! ! ! ! INT4 NOT NULL,! CONSTRAINT test_log_pk PRIMARY KEY(id, creation_date));CREATE RULE test_logging AS ON INSERT TO test! DO ALSO INSERT INTO test_log(id, creation_date, val) VALUES(NEW.id, NOW(), NEW.val);INSERT INTO test(id, count) VALUES (1, RANDOM());# SELECT * FROM test; # SELECT * FROM test_log; id | val id | creation_date! ! | val----+------ ----+--------------------------+----- 1 | 46228 1 | 2012-05-03 07:02:16.43841 15375(1 row) (1 row)
    • 28. Use Case
    • 29. reconciliations_datafinancial_operations idid bank_numamount start_countcard_number end_countcashier_id deposit_slip_numberreconciliation_id cashier_id
    • 30. reconciliations_datafinancial_operations idid bank_numamount start_countcard_number end_countcashier_id deposit_slip_numberreconciliation_id cashier_id
    • 31. CREATE OR REPLACE VIEW reconciliations AS SELECT ! rec.*, ! fin1.id AS start_id, ! fin1.upload_date AS start_date, ! fin2.id AS end_id, ! fin2.upload_date AS end_date FROM ! reconciliations_data rec LEFT OUTER JOIN ! (SELECT DISTINCT first_value(id) OVER (partition BY reconciliation_id ORDER BY upload_date ASC) AS id, first_value(upload_date) OVER (partition BY reconciliation_id ORDER BY upload_date ASC) AS upload_date, reconciliation_id FROM financial_operations) AS fin1 ON (fin1.reconciliation_id = rec.id) LEFT OUTER JOIN ! (SELECT DISTINCT first_value(id) OVER (partition BY reconciliation_id ORDER BY upload_date DESC) AS id, first_value(upload_date) OVER (partition BY reconciliation_id ORDER BY upload_date DESC) AS upload_date, reconciliation_id FROM financial_operations) AS fin2 ON (fin2.reconciliation_id = rec.id)
    • 32. reconciliations_datafinancial_operations idid bank_numamount start_countcard_number end_countcashier_id deposit_slip_numberreconciliation_id cashier_id
    • 33. reconciliations_data financial_operations id id bank_num amount start_count card_number end_count cashier_id deposit_slip_number reconciliation_id cashier_idCREATE OR REPLACE RULE fin_ops_from_recons AS ON INSERT TO reconciliations DO INSTEAD ( INSERT INTO reconciliations_data (cashier_id, lot_id, lane_id, start_count, end_count, cash_deposited, bank_amount,bank_returned, deposit_slip_number, created_at, updated_at) VALUES (new.cashier_id, new.lot_id, new.lane_id, new.start_count, new.end_count, new.cash_deposited, new.bank_amount,new.bank_returned, new.deposit_slip_number, new.created_at, new.updated_at) RETURNING! ! ! currval(reconciliations_id_seq)::integer,! ! ! cashier_id,! ! ! lot_id,! ! ! lane_id,! ! ! start_count,! ! ! end_count,! ! ! cash_deposited,! ! ! bank_amount,! ! ! bank_returned,! ! ! deposit_slip_number,! ! ! created_at,! ! ! updated_at,! ! ! null::varchar,! ! ! null::varchar,! ! ! null::timestamp,! ! ! null::varchar,! ! ! null::timestamp; ;
    • 34. reconciliations_datafinancial_operations idid bank_numamount start_countcard_number end_countcashier_id deposit_slip_numberreconciliation_id cashier_id
    • 35. reconciliations_data financial_operations id id bank_num amount start_count card_number end_count cashier_id deposit_slip_number reconciliation_id cashier_id UPDATE financial_operations SET reconciliation_id = lastval() WHERE financial_operations.remote_creation_date >= (( SELECT financial_operations.remote_creation_date FROM financial_operations WHERE financial_operations.id::text = new.start_id::text)) AND financial_operations.remote_creation_date <=(( SELECT financial_operations.remote_creation_date FROM financial_operations WHERE financial_operations.id::text = new.end_id::text)) AND financial_operations.order_transaction_id IS NOT NULLAND CASE WHEN new.lot_id IS NULL THEN new.cashier_id::text = (SELECT order_transactions.user_id FROM order_transactions WHERE order_transactions.id::text = financial_operations.order_transaction_id::text)::text ELSE ((new.cashier_id::text, new.lot_id) = ( SELECT order_transactions.user_id, order_transactions.lot_id FROM order_transactions WHERE order_transactions.id::text = financial_operations.order_transaction_id::text)) END; );
    • 36. FIN guyren@relevantlogic.com

    ×