Multi-Master Replication with Slony
Upcoming SlideShare
Loading in...5
×
 

Multi-Master Replication with Slony

on

  • 3,389 views

ne of the most sought after features in PostgreSQL is a scalable multi-master replication solution. While there does exists some tools to create multi-master clusters such as Bucardo and pgpool-II, ...

ne of the most sought after features in PostgreSQL is a scalable multi-master replication solution. While there does exists some tools to create multi-master clusters such as Bucardo and pgpool-II, they may not be the right fit for an application. In this session, you will learn some of the strengths and weaknesses of these more popular multi-master solutions for PostgreSQL and how they compare to using Slony for your multi-master needs. We will explore the types of deployments best suited for a Slony deployment and the steps necessary to configure a multi-master solution for PostgreSQL.

Statistics

Views

Total Views
3,389
Views on SlideShare
3,388
Embed Views
1

Actions

Likes
4
Downloads
74
Comments
0

1 Embed 1

https://duckduckgo.com 1

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Multi-Master Replication with Slony Multi-Master Replication with Slony Presentation Transcript

    • Multi-Master Replication Using Slony
    • Who Am I?
    • Jim Mlodgenski
      • Co-organizer of NYCPUG
      • Founder of Cirrus Technologies
      • Former Chief Architect of EnterpriseDB
    • What is Mult-Master?
    • Multiple phyiscal servers in a cluster allow for the updating of data
      • Increases Availability
      • Increases performance
        • Most noticable over a WAN
    • CAP Theorem
    • The theory put forth by Brewer that only 2 of the 3 are possible in a distributed system
      • C onsistency
      • A vailability
      • P artition Tolerance
    • Sync vs. Async
    • Synchronous
      • All transactions applied to all server before a success is returned to the client
        • Simpler to design and architect
        • At the expense of performance
    • Sync vs. Async
    • Asynchronous
      • A transaction is applied to a single server before a success is returned to the client
        • About the same performance as a single server
        • Need to deal with conflict resolution
    • Existing Solutions
    • Bucardo
    • PgPool
    • RubyRep
    • Bucardo
    • Asynchronous multi-master, master-slave, event based replication solution
      • Used in several production deployments
      • Limited to 2 masters
    • PgPool
    • Synchronous statement based, multi-master replication solution
      • Much more than a replication solution
      • Need to deal with Nondeterministic functions
    • RubyRep
    • Asynchronous multi-master, event based replication solution
      • Not a PostgreSQL only solution
      • Limited to 2 masters
    • What is Slony?
    • Asynchronous master-slave, event based replication solution
      • Proven production use cases
      • Cascading replication
    • Retail Store Problem
    • Corporate HQ controls the pricing
    • The stores control the daily sales information
    • Retail Store Problem
    • The information pushed down for HQ is exactly what Slony is good at
      • The tables controlled at HQ replicates to the many stores
      • The tables at the stores are read-only
    • Retail Store Problem
    • A single replication set can control this
      • May need to cascade if there are many stores
    slonik <<_EOF_ cluster name = HQ; node 1 admin conninfo = 'dbname=hq'; node 2 admin conninfo = 'dbname=store1'; node 3 admin conninfo = 'dbname=store2'; node 4 admin conninfo = 'dbname=store3'; node 5 admin conninfo = 'dbname=store4'; node 6 admin conninfo = 'dbname=store5'; init cluster ( id=1, comment = 'HQ Node'); create set (id=1, origin=1, comment='All HQ tables'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.items', comment='items table'); set add table (set id=1, origin=1, id=2, fully qualified name = 'public.prices', comment='prices table'); store node (id=2, comment = 'Store1 node', event node=1); store path (server = 1, client = 2, conninfo='dbname=hq'); store path (server = 2, client = 1, conninfo='dbname=store1'); ...
    • Retail Store Problem
    • Replicating all of the stores data to HQ is the challenge
      • Use table inheritance
    • Retail Store Problem
    • Each store has its one partition as well as the master partition
      • The appropriate triggers or rules should be applied to keep the structure transparent to the application
    • Retail Store Problem
    • A replication set is needed for each store
    • Cascading will be necessary with many stores
    slonik <<_EOF_ cluster name = Store1; node 1 admin conninfo = 'dbname=store1'; node 2 admin conninfo = 'dbname=hq'; init cluster ( id=1, comment = 'Store1 Node'); create set (id=1, origin=1, comment='All Store1 sales tables'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.sales_store1', comment='sales partition'); set add table (set id=1, origin=1, id=2, fully qualified name = 'public.sales_line_store1', comment='sales detail partition'); store node (id=2, comment = 'HQ node', event node=1); store path (server = 1, client = 2, conninfo='dbname=store1'); store path (server = 2, client = 1, conninfo='dbname=hq'); _EOF_
    • Retail Store Problem
    • Potential pitfalls
      • Many replication set and many slon daemons running
      • Need to deal with the complexity of table inheritance
    • Regional Office Problem
    • NY, London, Tokyo each control their own accounts
    • All accounts need to be visible
    • Changes to international accounts do occur
    • Regional Office Problem
    • Challenges
      • Need to deal with conflict resolution
      • Unique account identifier across all regions
      • Ping-pong effect
    • Regional Office Problem
    • The application needs to deal with the account table as it is designed but we need additional fields
      • SELECT *
      • FROM accounts
      • WHERE account_no = X
    • Regional Office Problem
    • Create a table with the necessary origin
    • Use a view to mask the field
      • CREATE VIEW accounts AS
      • SELECT account_id, account_no, account_name,
      • amount, created_date, last_update
      • FROM accounts_tbl
    • Regional Office Problem
    • Need a central broker to handle distribution of the changes
      • One place for conflict resolution
    slonik <<_EOF_ cluster name = CENTRAL_BROKER; node 1 admin conninfo = 'dbname=london'; node 2 admin conninfo = 'dbname=newyork'; node 3 admin conninfo = 'dbname=tokyo'; init cluster ( id=1, comment = 'Central Broker'); create set (id=1, origin=1, comment='Account Table'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.accounts_tbl', comment='accounts table'); store node (id=2, comment = 'New York', event node=1); store path (server = 1, client = 2, conninfo='dbname=london'); store path (server = 2, client = 1, conninfo='dbname=newyork'); store node (id=3, comment = 'Tokyo', event node=1); store path (server = 1, client = 3, conninfo='dbname=london'); store path (server = 3, client = 1, conninfo='dbname=tokyo');
    • Regional Office Problem
    • Use table inheritance to handle the changes in the offices
      • No account_id on the shadow tables
    • Regional Office Problem
    • Adding rules to the accounts view allows the transactions to occur on the shadow tables
    CREATE OR REPLACE RULE accounts_i AS ON INSERT TO accounts DO INSTEAD INSERT INTO accounts_shadow_newyork VALUES (NEW.account_no, NEW.account_name, NEW.amount, NEW.created_date, NEW.last_update, 'newyork')
    • Regional Office Problem
    • Replicate the local transactions to the central broker
    slonik <<_EOF_ cluster name = NEWYORK; node 1 admin conninfo = 'dbname=newyork'; node 2 admin conninfo = 'dbname=london'; init cluster ( id=1, comment = 'New York'); create set (id=1, origin=1, comment='Account Shadow Table'); set add table (set id=1, origin=1, id=1, fully qualified name = 'public.accounts_shadow_newyork'); store node (id=2, comment = 'New York', event node=1); store path (server = 1, client = 2, conninfo='dbname=newyork'); store path (server = 2, client = 1, conninfo='dbname=london'); _EOF_
  • CREATE OR REPLACE FUNCTION accounts_shadow_trig() RETURNS trigger AS $BODY$ DECLARE existing_account_id integer; BEGIN -- Business Logic for First-In Wins SELECT account_id INTO existing_account_id FROM accounts_tbl WHERE account_no = NEW.account_no; IF FOUND THEN RAISE INFO 'Account % already exists. Ignoring new INSERT', NEW.account_no; ELSE INSERT INTO accounts_tbl VALUES (nextval('account_id_seq'), NEW.account_no, NEW.account_name, NEW.amount, NEW.created_date, NEW.last_update, NEW.origin_code); END IF; RETURN NEW; END; $BODY$ LANGUAGE plpgsql;
      Regional Office Problem
    • Add the conflict resolution logic to the broker as triggers on the shadow tables
    • Regional Office Problem
    • Potential pitfalls
      • Many moving parts to maintenance is a heavy burden
      • Reliance on a central broker
    • Moral of the Story
    • Slony is extremely flexible
    • PostgreSQL is extremely flexible
    • Together you can do some strange and powerful things
    • Don't use multi-master unless absolutely necessary
    • Questions?
      Jim Mlodgenski
        Email: [email_address] Twitter: @jim_mlodgenski