<ul>Multi-Master Replication Using Slony </ul>
<ul>Who Am I? </ul><ul><li>Jim Mlodgenski </li><ul><li>Co-organizer of NYCPUG
Founder of Cirrus Technologies
Former Chief Architect of EnterpriseDB </li></ul></ul>
<ul>What is Mult-Master? </ul><ul><li>Multiple phyiscal servers in a cluster allow for the updating of data </li><ul><li>I...
Increases performance </li><ul><li>Most noticable over a WAN </li></ul></ul></ul>
<ul>CAP Theorem </ul><ul><li>The theory put forth by Brewer that only 2 of the 3 are possible in a distributed system </li...
A vailability
P artition Tolerance </li></ul></ul>
<ul>Sync vs. Async </ul><ul><li>Synchronous </li><ul><li>All transactions applied to all server before a success is return...
At the expense of performance </li></ul></ul></ul>
<ul>Sync vs. Async </ul><ul><li>Asynchronous </li><ul><li>A transaction is applied to a single server before a success is ...
Need to deal with conflict resolution </li></ul></ul></ul>
<ul>Existing Solutions </ul><ul><li>Bucardo
PgPool
RubyRep </li></ul>
<ul>Bucardo </ul><ul><li>Asynchronous multi-master, master-slave, event based replication solution </li><ul><li>Used in se...
Limited to 2 masters  </li></ul></ul>
<ul>PgPool </ul><ul><li>Synchronous statement based, multi-master replication solution </li><ul><li>Much more than a repli...
Need to deal with Nondeterministic functions </li></ul></ul>
<ul>RubyRep </ul><ul><li>Asynchronous multi-master, event based replication solution </li><ul><li>Not a PostgreSQL only so...
Limited to 2 masters </li></ul></ul>
<ul>What is Slony? </ul><ul><li>Asynchronous master-slave, event based replication solution </li><ul><li>Proven production...
Cascading replication </li></ul></ul>
<ul>Retail Store Problem </ul><ul><li>Corporate HQ controls the pricing
The stores  control the daily sales information </li></ul>
<ul>Retail Store Problem </ul><ul><li>The information pushed down for HQ is exactly what Slony is good at </li><ul><li>The...
The tables at the stores are read-only </li></ul></ul>
<ul>Retail Store Problem </ul><ul><li>A single replication set can control this </li><ul><li>May need to cascade if there ...
<ul>Retail Store Problem </ul><ul><li>Replicating all of the stores data to HQ is the challenge </li><ul><li>Use table inh...
Upcoming SlideShare
Loading in...5
×

Multi-Master Replication with Slony

3,813

Published on

ne of the most sought after features in PostgreSQL is a scalable multi-master replication solution. While there does exists some tools to create multi-master clusters such as Bucardo and pgpool-II, they may not be the right fit for an application. In this session, you will learn some of the strengths and weaknesses of these more popular multi-master solutions for PostgreSQL and how they compare to using Slony for your multi-master needs. We will explore the types of deployments best suited for a Slony deployment and the steps necessary to configure a multi-master solution for PostgreSQL.

Published in: Technology

Multi-Master Replication with Slony

  1. 1. <ul>Multi-Master Replication Using Slony </ul>
  2. 2. <ul>Who Am I? </ul><ul><li>Jim Mlodgenski </li><ul><li>Co-organizer of NYCPUG
  3. 3. Founder of Cirrus Technologies
  4. 4. Former Chief Architect of EnterpriseDB </li></ul></ul>
  5. 5. <ul>What is Mult-Master? </ul><ul><li>Multiple phyiscal servers in a cluster allow for the updating of data </li><ul><li>Increases Availability
  6. 6. Increases performance </li><ul><li>Most noticable over a WAN </li></ul></ul></ul>
  7. 7. <ul>CAP Theorem </ul><ul><li>The theory put forth by Brewer that only 2 of the 3 are possible in a distributed system </li><ul><li>C onsistency
  8. 8. A vailability
  9. 9. P artition Tolerance </li></ul></ul>
  10. 10. <ul>Sync vs. Async </ul><ul><li>Synchronous </li><ul><li>All transactions applied to all server before a success is returned to the client </li><ul><li>Simpler to design and architect
  11. 11. At the expense of performance </li></ul></ul></ul>
  12. 12. <ul>Sync vs. Async </ul><ul><li>Asynchronous </li><ul><li>A transaction is applied to a single server before a success is returned to the client </li><ul><li>About the same performance as a single server
  13. 13. Need to deal with conflict resolution </li></ul></ul></ul>
  14. 14. <ul>Existing Solutions </ul><ul><li>Bucardo
  15. 15. PgPool
  16. 16. RubyRep </li></ul>
  17. 17. <ul>Bucardo </ul><ul><li>Asynchronous multi-master, master-slave, event based replication solution </li><ul><li>Used in several production deployments
  18. 18. Limited to 2 masters </li></ul></ul>
  19. 19. <ul>PgPool </ul><ul><li>Synchronous statement based, multi-master replication solution </li><ul><li>Much more than a replication solution
  20. 20. Need to deal with Nondeterministic functions </li></ul></ul>
  21. 21. <ul>RubyRep </ul><ul><li>Asynchronous multi-master, event based replication solution </li><ul><li>Not a PostgreSQL only solution
  22. 22. Limited to 2 masters </li></ul></ul>
  23. 23. <ul>What is Slony? </ul><ul><li>Asynchronous master-slave, event based replication solution </li><ul><li>Proven production use cases
  24. 24. Cascading replication </li></ul></ul>
  25. 25. <ul>Retail Store Problem </ul><ul><li>Corporate HQ controls the pricing
  26. 26. The stores control the daily sales information </li></ul>
  27. 27. <ul>Retail Store Problem </ul><ul><li>The information pushed down for HQ is exactly what Slony is good at </li><ul><li>The tables controlled at HQ replicates to the many stores
  28. 28. The tables at the stores are read-only </li></ul></ul>
  29. 29. <ul>Retail Store Problem </ul><ul><li>A single replication set can control this </li><ul><li>May need to cascade if there are many stores </li></ul></ul>slonik <<_EOF_ cluster name = HQ; node 1 admin conninfo = 'dbname=hq'; node 2 admin conninfo = 'dbname=store1'; node 3 admin conninfo = 'dbname=store2'; node 4 admin conninfo = 'dbname=store3'; node 5 admin conninfo = 'dbname=store4'; node 6 admin conninfo = 'dbname=store5'; init cluster ( id=1, comment = 'HQ Node'); create set (id=1, origin=1, comment='All HQ tables'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.items', comment='items table'); set add table (set id=1, origin=1, id=2, fully qualified name = 'public.prices', comment='prices table'); store node (id=2, comment = 'Store1 node', event node=1); store path (server = 1, client = 2, conninfo='dbname=hq'); store path (server = 2, client = 1, conninfo='dbname=store1'); ...
  30. 30. <ul>Retail Store Problem </ul><ul><li>Replicating all of the stores data to HQ is the challenge </li><ul><li>Use table inheritance </li></ul></ul>
  31. 31. <ul>Retail Store Problem </ul><ul><li>Each store has its one partition as well as the master partition </li><ul><li>The appropriate triggers or rules should be applied to keep the structure transparent to the application </li></ul></ul>
  32. 32. <ul>Retail Store Problem </ul><ul><li>A replication set is needed for each store
  33. 33. Cascading will be necessary with many stores </li></ul>slonik <<_EOF_ cluster name = Store1; node 1 admin conninfo = 'dbname=store1'; node 2 admin conninfo = 'dbname=hq'; init cluster ( id=1, comment = 'Store1 Node'); create set (id=1, origin=1, comment='All Store1 sales tables'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.sales_store1', comment='sales partition'); set add table (set id=1, origin=1, id=2, fully qualified name = 'public.sales_line_store1', comment='sales detail partition'); store node (id=2, comment = 'HQ node', event node=1); store path (server = 1, client = 2, conninfo='dbname=store1'); store path (server = 2, client = 1, conninfo='dbname=hq'); _EOF_
  34. 34. <ul>Retail Store Problem </ul><ul><li>Potential pitfalls </li><ul><li>Many replication set and many slon daemons running
  35. 35. Need to deal with the complexity of table inheritance </li></ul></ul>
  36. 36. <ul>Regional Office Problem </ul><ul><li>NY, London, Tokyo each control their own accounts
  37. 37. All accounts need to be visible
  38. 38. Changes to international accounts do occur </li></ul>
  39. 39. <ul>Regional Office Problem </ul><ul><li>Challenges </li><ul><li>Need to deal with conflict resolution
  40. 40. Unique account identifier across all regions
  41. 41. Ping-pong effect </li></ul></ul>
  42. 42. <ul>Regional Office Problem </ul><ul><li>The application needs to deal with the account table as it is designed but we need additional fields </li><ul><li>SELECT *
  43. 43. FROM accounts
  44. 44. WHERE account_no = X </li></ul></ul>
  45. 45. <ul>Regional Office Problem </ul><ul><li>Create a table with the necessary origin
  46. 46. Use a view to mask the field </li></ul><ul><ul><li>CREATE VIEW accounts AS
  47. 47. SELECT account_id, account_no, account_name,
  48. 48. amount, created_date, last_update
  49. 49. FROM accounts_tbl </li></ul></ul>
  50. 50. <ul>Regional Office Problem </ul><ul><li>Need a central broker to handle distribution of the changes </li><ul><li>One place for conflict resolution </li></ul></ul>slonik <<_EOF_ cluster name = CENTRAL_BROKER; node 1 admin conninfo = 'dbname=london'; node 2 admin conninfo = 'dbname=newyork'; node 3 admin conninfo = 'dbname=tokyo'; init cluster ( id=1, comment = 'Central Broker'); create set (id=1, origin=1, comment='Account Table'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.accounts_tbl', comment='accounts table'); store node (id=2, comment = 'New York', event node=1); store path (server = 1, client = 2, conninfo='dbname=london'); store path (server = 2, client = 1, conninfo='dbname=newyork'); store node (id=3, comment = 'Tokyo', event node=1); store path (server = 1, client = 3, conninfo='dbname=london'); store path (server = 3, client = 1, conninfo='dbname=tokyo');
  51. 51. <ul>Regional Office Problem </ul><ul><li>Use table inheritance to handle the changes in the offices </li></ul><ul><ul><li>No account_id on the shadow tables </li></ul></ul>
  52. 52. <ul>Regional Office Problem </ul><ul><li>Adding rules to the accounts view allows the transactions to occur on the shadow tables </li></ul>CREATE OR REPLACE RULE accounts_i AS ON INSERT TO accounts DO INSTEAD INSERT INTO accounts_shadow_newyork VALUES (NEW.account_no, NEW.account_name, NEW.amount, NEW.created_date, NEW.last_update, 'newyork')
  53. 53. <ul>Regional Office Problem </ul><ul><li>Replicate the local transactions to the central broker </li></ul>slonik <<_EOF_ cluster name = NEWYORK; node 1 admin conninfo = 'dbname=newyork'; node 2 admin conninfo = 'dbname=london'; init cluster ( id=1, comment = 'New York'); create set (id=1, origin=1, comment='Account Shadow Table'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.accounts_shadow_newyork'); store node (id=2, comment = 'New York', event node=1); store path (server = 1, client = 2, conninfo='dbname=newyork'); store path (server = 2, client = 1, conninfo='dbname=london'); _EOF_
  54. 54. CREATE OR REPLACE FUNCTION accounts_shadow_trig() RETURNS trigger AS $BODY$ DECLARE existing_account_id integer; BEGIN -- Business Logic for First-In Wins SELECT account_id INTO existing_account_id FROM accounts_tbl WHERE account_no = NEW.account_no; IF FOUND THEN RAISE INFO 'Account % already exists. Ignoring new INSERT', NEW.account_no; ELSE INSERT INTO accounts_tbl VALUES (nextval('account_id_seq'), NEW.account_no, NEW.account_name, NEW.amount, NEW.created_date, NEW.last_update, NEW.origin_code); END IF; RETURN NEW; END; $BODY$ LANGUAGE plpgsql; <ul>Regional Office Problem </ul><ul><li>Add the conflict resolution logic to the broker as triggers on the shadow tables </li></ul>
  55. 55. <ul>Regional Office Problem </ul><ul><li>Potential pitfalls </li><ul><li>Many moving parts to maintenance is a heavy burden
  56. 56. Reliance on a central broker </li></ul></ul>
  57. 57. <ul>Moral of the Story </ul><ul><li>Slony is extremely flexible
  58. 58. PostgreSQL is extremely flexible
  59. 59. Together you can do some strange and powerful things
  60. 60. Don't use multi-master unless absolutely necessary </li></ul>
  61. 61. <ul>Questions? </ul><ul>Jim Mlodgenski <ul>Email: [email_address] Twitter: @jim_mlodgenski </ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×