Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PostgreSQL10の新機能 ~ロジカルレプリケーションを中心に~

139 views

Published on

2018-01-27 OSC 大阪で発表したものです。
マルチカラム統計情報についても触れています。

Published in: Software
  • Be the first to comment

  • Be the first to like this

PostgreSQL10の新機能 ~ロジカルレプリケーションを中心に~

  1. 1. PostgreSQL10 PostgreSQL 2018.1.27 2018 Osaka
  2. 2. ( ) PostgreSQL / Hadoop OpenStack Swift
  3. 3. PostgreSQL PostgreSQL 10 ( ) ( )
  4. 4. PostgreSQL RDBMS Ingres(1970~) PostgreSQL6.0(1996 ) 20 BSD (PostgreSQL ) PostgreSQL Global Development Group 4
  5. 5. PostgreSQL 10 2017/10/5 10.0 10.1
  6. 6. PostgreSQL 10 / 10 10.1 9.6 9.6.6 : DB :
  7. 7. PostgreSQL 10
  8. 8. (Petr Jelinek)
  9. 9. ... 9.0 DB Pacemaker HA &
  10. 10. 10.0 table DB PostgreSQL ( )
  11. 11. ~cont -> publication subscription ( pub, sub )
  12. 12. table 2 table 4 table 1 table 3 table 5 pub1 pub2 table 4 table 2 table 1 table 3 sub1(:pub1) sub2(:pub2) DB1 DB2 : 1 DB3 INSERT
  13. 13. : 1 =# CREATE PUBLICATION pub1 FOR TABLE table1, table2, table3 WITH (publish = ‘INSERT’); =# CREATE SUBSCRIPTION sub1 CONNECTION ‘host=[IP ] dbname=[DB ]’ PUBLICATION pub1; table 2 table 4 table 1 table 3 table 5 pub1 table 2 table 1 table 3 sub1(:pub1) DB1 DB2 INSERT
  14. 14. table 2 table 4 table 1 table 3 table 5 pub1 pub2 table 3 table 4 table 2 table 1 sub1(:pub1, pub2) sub2(:pub2) DB3 table 4 table 3 DB1 DB2 : 2
  15. 15. : 2 =# CREATE PUBLICATION pub1 FOR TABLE table1, table2, table3; =# CREATE PUBLICATION pub2 FOR TABLE table3, table4; =# CREATE SUBSCRIPTION sub1 CONNECTION ‘host=[IP ] dbname=[DB ]’ PUBLICATION pub1, pub2; table 2 table 4 table 1 table 3 table 5 pub2 table 1 DB2 table 3 pub1 sub1(:pub1, pub2) DB1 table 2 table 4
  16. 16. pub walsender (WAL) COMMIT pub sub ABORT sub sub apply worker walsender
  17. 17. INSERT WAL wal sender INSERT X INSERT Y INSERT Y (WAL) UPDATE A DELETE B INSERT C INSERT Z Pub DB apply worker table table table Sub DB
  18. 18. COMMIT WAL wal sender INSERT X INSERT Y COMMIT (WAL) UPDATE A DELETE B INSERT C INSERT Z Pub DB apply worker table table table Sub DB COMMIT Sub Sub
  19. 19. ABORT WAL wal sender INSERT X INSERT Y ABORT (WAL) UPDATE A DELETE B INSERT C INSERT Z Pub DB apply worker table table table Sub DB ABORT
  20. 20. COPY COPY sub table sync worker max_sync_workers_per_subscription ( :2)
  21. 21. sub
  22. 22. Sub ERROR: duplicate key value violates unique constraint "t1_pkey" DETAIL: Key (id)=(2) already exists. ... LOG: worker process: logical replication worker for subscription 16392 (PID 1494) exited with exit code 1 5
  23. 23. sub sub (LSN) pg_replication_origin_advance() pub pg_current_wal_lsn() pg_replication_origin_advance() pub pub sub
  24. 24. DDL CREATE TABLE TRUNCATE (serial, identity) pub serial, identity sub INSERT pub
  25. 25. COMMIT Sub COPY
  26. 26. ((Tomas Vondra, David Rowley, Álvaro Herrera)
  27. 27. DB . selectivity(WHERE )
  28. 28. 1 "WEHRE X = 0” * selectivity(X = 0) 9.6
  29. 29. 1 "WHERE (X = 0) AND (Y = 1)" * selectivity(X = 0) * selectivity(Y = 1) 9.6
  30. 30. 1 "WHERE (X = 0) AND (Y = 1)" * selectivity(X = 0) * selectivity(Y = 1) 9.6 X Y X Y
  31. 31. 3 9.6 x | y | z ---+---+--- 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 2 | 2 | 2 2 | 2 | 2 ….cont =# EXPLAIN ANALYZE SELECT * FROM test WHERE (X = 1) AND (Y = 1) AND (Z = 1)’; ———————————————- Seq Scan on test (cost=0.00..2.75 rows=1 width=12) (actual time=0.035..0.063 rows=10 loops=1) Filter: ((x = 1) AND (y = 1) AND (z = 1)) Rows Removed by Filter: 90 Planning time: 0.690 ms Execution time: 0.262 ms (5 rows)
  32. 32. 9.6 =# EXPLAIN ANALYZE SELECT * FROM test WHERE (X = 1) AND (Y = 1) AND (Z = 1)’; ———————————————- Seq Scan on test (cost=0.00..2.75 rows=1 width=12) (actual time=0.035..0.063 rows=10 loops=1) Filter: ((x = 1) AND (y = 1) AND (z = 1)) Rows Removed by Filter: 90 Planning time: 0.690 ms Execution time: 0.262 ms (5 rows) 10 1 3 x | y | z ---+---+--- 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 0 | 0 | 0 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 1 | 1 | 1 2 | 2 | 2 2 | 2 | 2 ….cont
  33. 33. =# CREATE STATISTICS stat_test ON x, y, z FROM test; =# ANALYZE; =# EXPLAIN ANALYZE SELECT * FROM test WHERE (X = 1) AND (Y = 1) AND (Z = 1); ———————————————- Seq Scan on test (cost=0.00..2.75 rows=10 width=12) (actual time=0.028..0.053 rows=10 loops=1) Filter: ((x = 1) AND (y = 1) AND (z = 1)) Rows Removed by Filter: 90 Planning time: 0.687 ms Execution time: 0.123 ms (5 rows)
  34. 34. CREATE STATISTICS =# SELECT * FROM pg_statistic_ext; stxrelid | 16607 stxname | stat_test stxnamespace | 2200 stxowner | 10 stxkeys | 1 2 3 stxkind | {d,f} stxndistinct | {"1, 2": 11, "1, 3": 11, "2, 3": 11, "1, 2, 3": 11} stxdependencies | {"1 => 2": 1.000000, "1 => 3": 1.000000, "2 => 1": 1.000000, "2 => 3": 1.000000, "3 => 1": 1.000000, "3 => 2": 1.000000, "1, 2 => 3": 1.000000, "1, 3 => 2": 1.000000, "2, 3 => 1": 1.000000} stxndistinct: stxdependencies:
  35. 35. CREATE STATISTICS =# SELECT * FROM pg_statistic_ext; stxrelid | 16607 stxname | stat_test stxnamespace | 2200 stxowner | 10 stxkeys | 1 2 3 stxkind | {d,f} stxndistinct | {"1, 2": 11, "1, 3": 11, "2, 3": 11, "1, 2, 3": 11} stxdependencies | {"1 => 2": 1.000000, "1 => 3": 1.000000, "2 => 1": 1.000000, "2 => 3": 1.000000, "3 => 1": 1.000000, "3 => 2": 1.000000, "1, 2 => 3": 1.000000, "1, 3 => 2": 1.000000, "2, 3 => 1": 1.000000}
  36. 36. ANALYZE CREATE STATISTICS
  37. 37. (Amit Langote)
  38. 38. : 1 AP 1 / table table table table
  39. 39. 9.6 CHECK
  40. 40. 9.6 table CREATE TABLE japan ( pref text, city text, data text ); table CREATE TABLE osaka ( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE TABLE kyoto ( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE TABLE shiga( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE TABLE nara ( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE OR REPLACE FUNCTION pref_insert_trigger_func() RETURNS TRIGGER AS $$ BEGIN IF ( NEW.pref = ' ') THEN INSERT INTO osaka VALUES (NEW.*); ELSIF ( NEW.pref = ' ') THEN INSERT INTO kyoto VALUES (NEW.*); ELSIF ( NEW.pref = ' ') THEN INSERT INTO shiga VALUES (NEW.*); ELSIF ( NEW.pref = ' ') THEN INSERT INTO nara VALUES (NEW.*); ELSE RAISE EXCEPTION 'Data out of range. Fix the pref_insert_trigger()'; END IF; RETURN NULL; END; $$ LANGUAGE plpgsql; table CREATE TRIGGER pref_insert_trigger BEFORE INSERT ON japan FOR EACH ROW EXECUTE PROCEDURE pref_insert_trigger_func();
  41. 41. 9.6 table CREATE TABLE japan ( pref text, city text, data text ); table CREATE TABLE osaka ( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE TABLE kyoto ( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE TABLE shiga( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE TABLE nara ( CHECK (pref IN (' ')) ) INHERITS (japan); CREATE OR REPLACE FUNCTION pref_insert_trigger_func() RETURNS TRIGGER AS $$ BEGIN IF ( NEW.pref = ' ') THEN INSERT INTO osaka VALUES (NEW.*); ELSIF ( NEW.pref = ' ') THEN INSERT INTO kyoto VALUES (NEW.*); ELSIF ( NEW.pref = ' ') THEN INSERT INTO shiga VALUES (NEW.*); ELSIF ( NEW.pref = ' ') THEN INSERT INTO nara VALUES (NEW.*); ELSE RAISE EXCEPTION 'Data out of range. Fix the pref_insert_trigger()'; END IF; RETURN NULL; END; $$ LANGUAGE plpgsql; table CREATE TRIGGER pref_insert_trigger BEFORE INSERT ON japan FOR EACH ROW EXECUTE PROCEDURE pref_insert_trigger_func(); trigger
  42. 42. 9.6
  43. 43. 10 table
  44. 44. 10 table -- table CREATE TABLE japan ( pref text, city text, data text ) PARTITION BY LIST (pref); table CREATE TABLE osaka PARTITION OF japan FOR VALUES IN (' '); CREATE TABLE kyoto PARTITION OF japan FOR VALUES IN (' '); CREATE TABLE shiga PARTITION OF japan FOR VALUES IN (' '); CREATE TABLE nara PARTITION OF japan FOR VALUES IN (' '); trigger
  45. 45. 10 INSERT 10 0 35 70 105 140 elaped time [s] trigger based native 300
  46. 46. 10 UPDATE DELETE INSERT 11
  47. 47. (Robert Haas, Amit Kapila, Rahila Syed, Rafia Sabih, Dilip Kumar)
  48. 48. 1 9.6 10
  49. 49. 9.6
  50. 50. 9.6 ... Scan Sequential Scan Join Nested Loop Join, Hash Join
  51. 51. 10 ... Scan Sequential Scan, Bitmap Heap Scan, Index Scan Join Nested Loop Join, Hash Join, Merge Join
  52. 52. (TPC-H) https://www.pgcon.org/2017/schedule/attachments/445_Next-Generation%20Parallel%20Query%20-%20PGCon.pdf
  53. 53. parallel_setup_cost, parallel_tuple_cost 0
  54. 54. PostgreSQL 10 https://www.slideshare.net/toshiharada/20171106-ntttxpostgre-sql10-81645645 PostgreSQL10 https://www.slideshare.net/masahikosawada98/postgresql10 Logical Replication Internals https://www.slideshare.net/noriyoshishinoda/pgconfasia-2017-logical-replication- internals-japanese Next-Generation Parallel Query https://www.pgcon.org/2017/schedule/attachments/445_Next-Generation Parallel Query - PGCon.pdf

×