Slony-I Replication for
Asynchronous
PostgreSQL
                     Slony-I on Microsoft Windows

                       ...
Slony-I


      Asynchronous Replication for PostgreSQL

      Developed by Jan Wieck of Afilias

      Now a community pr...
Uses

      Load balancing

      Redundancy/Failover

      Remote/distributed servers

      Upgrades

28th October 2005...
Architecture




28th October 2005   Dave Page - Magnus Hagander - Andreas Pflug   4
Flexible Topology




28th October 2005   Dave Page - Magnus Hagander - Andreas Pflug   5
Object types


      Data

      Sequence values

      Schema changes



28th October 2005   Dave Page - Magnus Hagander ...
Porting team

      Hiroshi Saito
             pgAdmin, psqlODBC, Npgsql, pgInstaller

      Dave Page
            pgAdmin...
Hiroshi



      Provided the initial ‘quick n dirty’ port.

      Organised the project




28th October 2005   Dave Page...
Dave

      ‘Ducttape’ test suite

      New regression test suite

      Build system/Makefiles

      Patch/CVS manageme...
Magnus

      Code port

            slonik – based on Hiroshi’s work

            slon
                    Reuse pgpipe f...
Andreas




      GUI Management using pgAdmin




28th October 2005   Dave Page - Magnus Hagander - Andreas Pflug   11
Slony-I Replication for
Asynchronous
PostgreSQL
                      Porting Slony-I

                       Magnus Hagan...
Porting overview

      The bad
            Designed for Unix
            Relied on Unix tools and architecture
      The ...
Porting Slonik - easy

      No shell utils available
            Slonik 1.1 uses SED
            Hiroshi already fixed
  ...
Porting Slon - easy

      Pthreads
            Find library to link with
      Winsock
            Simple initialization ...
Porting slon – a bit more work

      Eventlog integration
            Centralised logging already
            Eventlog wh...
Porting slon – most work

      fork()
      Service integration
      Two problems, one solution




28th October 2005   ...
Slon – Unix architecture

                                       init

              rc.slony                             ...
Slon – Windows architecture

                    Service Control Manager
                              init

             ...
Porting – end result

      Slon runs on the commandline
            Only for testing/debugging!
      Single service, mul...
Simple slony replication




                         DEMO


28th October 2005   Dave Page - Magnus Hagander - Andreas Pfl...
Base table creation
  db1
  CREATE TABLE t (
      name text NOT NULL PRIMARY KEY)
  INSERT INTO t VALUES (’Dave’)
  INSER...
Installing Slony
  slon       –regservice
  slon       –addengine c:slonydb1.conf
  slon       –addengine c:slonydb2.conf
...
Slon setup script
  # Create slony cluster
  cluster name = test;
  node 1 admin conninfo = ’host=127.0.0.1
      user=pos...
Slon setup script (contd)
  # Create node for second engine
  store node (id=2, comment=’Node 2’);
  # Create paths betwee...
Slony-I Replication for
Asynchronous
PostgreSQL
           Graphical management of Slony-I

                           And...
pgAdmin III architecture

      C++
      wxWidgets 2.6
      Native libpq PostgreSQL connection
      Native Windows and ...
Slony-I installation: modules

      Performed by Windows installer
      Performed by make;make install from
      source...
Slony-I installation: Create cluster

      First node in cluster
      Node: database with
      installed cluster and
  ...
Slony-I installation: Join cluster

      Create node and
      copy replication
      configuration from
      existing n...
Slony-I installation: paths

      Path: describes how
      a slon process
      connects to other
      nodes
      Libp...
Slony-I installation: listens

      Listen: instructs a
      node to poll events
      from other nodes




28th October...
Slony-I cluster status

      See node statistics




28th October 2005   Dave Page - Magnus Hagander - Andreas Pflug   33
Slony-I replication sets

      Set: collection of
      tables and sequences
      All table and
      sequence data
    ...
Slony-I Tables

      Table needs unique
      index, PK preferred
      pgAdmin doesn‘t
      offer tables without
      ...
Slony-I subscriptions

      Table and sequences
      must be present in
      slave node before
      subscribing!
     ...
Slony-I DDL script replication

      Use replication to
      execute changes to
      subscribed tables to
      insure ...
Slony-I switch over

      Gracefully exchange master and slave
      role to a set between two nodes
      Both nodes mus...
Slony-I fail over

      Master node has failed
      Failover tries to restore as much data
      from slave nodes as pos...
pgAdmin future

      Coming in V1.6:
            Slony-I failover support
            Set creation wizard
            Hea...
Slony-I and pgAdmin Conclusion

      Most Slony-I functions accessible
      through easy-to-use GUI
      At-a-glance vi...
Upcoming SlideShare
Loading in …5
×

Replicating PostgreSQL Databases Using Slony-I

5,288 views

Published on

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,288
On SlideShare
0
From Embeds
0
Number of Embeds
33
Actions
Shares
0
Downloads
173
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Replicating PostgreSQL Databases Using Slony-I

  1. 1. Slony-I Replication for Asynchronous PostgreSQL Slony-I on Microsoft Windows Dave Page 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 1
  2. 2. Slony-I Asynchronous Replication for PostgreSQL Developed by Jan Wieck of Afilias Now a community project 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 2
  3. 3. Uses Load balancing Redundancy/Failover Remote/distributed servers Upgrades 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 3
  4. 4. Architecture 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 4
  5. 5. Flexible Topology 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 5
  6. 6. Object types Data Sequence values Schema changes 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 6
  7. 7. Porting team Hiroshi Saito pgAdmin, psqlODBC, Npgsql, pgInstaller Dave Page pgAdmin, PostgreSQL, psqlODBC, pgInstaller, Npgsql, pgWeb Magnus Hagander PostgreSQL Server, pgInstaller, pgWeb Andreas Pflug pgAdmin, PostgreSQL Server 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 7
  8. 8. Hiroshi Provided the initial ‘quick n dirty’ port. Organised the project 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 8
  9. 9. Dave ‘Ducttape’ test suite New regression test suite Build system/Makefiles Patch/CVS management 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 9
  10. 10. Magnus Code port slonik – based on Hiroshi’s work slon Reuse pgpipe from PostgreSQL Service control code Event logging 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 10
  11. 11. Andreas GUI Management using pgAdmin 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 11
  12. 12. Slony-I Replication for Asynchronous PostgreSQL Porting Slony-I Magnus Hagander 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 12
  13. 13. Porting overview The bad Designed for Unix Relied on Unix tools and architecture The good Portable between Unixes Based on PostgreSQL build system 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 13
  14. 14. Porting Slonik - easy No shell utils available Slonik 1.1 uses SED Hiroshi already fixed Path issues Find the ”share” directory Windows compatible paths 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 14
  15. 15. Porting Slon - easy Pthreads Find library to link with Winsock Simple initialization issue Pipes Steal from PostgreSQL Signals Ignore! 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 15
  16. 16. Porting slon – a bit more work Eventlog integration Centralised logging already Eventlog when service, stdout when console Create message library Versioning metadata Steal most from PostgreSQL Decimal version number in config.h 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 16
  17. 17. Porting slon – most work fork() Service integration Two problems, one solution 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 17
  18. 18. Slon – Unix architecture init rc.slony rc.xyz slon watchdog slon watchdog fork() fork() slon engine slon engine 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 18
  19. 19. Slon – Windows architecture Service Control Manager init rc.slony rc.xyz slon slon win32 service handler watchdog slon watchdog fork() CreateProcess() CreateProcess() fork() slon engine slon engine 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 19
  20. 20. Porting – end result Slon runs on the commandline Only for testing/debugging! Single service, multiple engines One config file per engine Paths stored in registry, add/remove with slon commandline Multiple services, multiple engines Different versions of slon 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 20
  21. 21. Simple slony replication DEMO 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 21
  22. 22. Base table creation db1 CREATE TABLE t ( name text NOT NULL PRIMARY KEY) INSERT INTO t VALUES (’Dave’) INSERT INTO t VALUES (’Magnus’) db2 CREATE TABLE t ( name text NOT NULL PRIMARY KEY) 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 22
  23. 23. Installing Slony slon –regservice slon –addengine c:slonydb1.conf slon –addengine c:slonydb2.conf slon -listengines db1.conf log_level=1 log_timestamp=false cluster_name='test' conn_info='host=127.0.0.1 user=postgres dbname=db1' 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 23
  24. 24. Slon setup script # Create slony cluster cluster name = test; node 1 admin conninfo = ’host=127.0.0.1 user=postgres database=db1’; node 2 admin conninfo = ’host=127.0.0.1 user=postgres database=db2’; init cluster (id=1, comment=’Node 1’) # Create set of tables with one table create set (id=1, origin=1) set add table (set id=1, origin=1, id=1, fully qualified name = ’public.t’) 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 24
  25. 25. Slon setup script (contd) # Create node for second engine store node (id=2, comment=’Node 2’); # Create paths between the two nodes store path (server=1,client=2, conninfo=’host=127.0.0.1 user=postgres dbname=db1’); store path (server=2,client=1, conninfo=’host=127.0.0.1 user=postgres dbname=db2’); store listen (origin=1, provider=1, receiver=2); store listen (origin=2, provider=2, receiver=1); # Subscribe the slave subscribe set (id=1, provider=1, receiver=2, forward=no) 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 25
  26. 26. Slony-I Replication for Asynchronous PostgreSQL Graphical management of Slony-I Andreas Pflug 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 26
  27. 27. pgAdmin III architecture C++ wxWidgets 2.6 Native libpq PostgreSQL connection Native Windows and GTK2 look and feel For PostgreSQL 7.3 and above Some helper programs and modules 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 27
  28. 28. Slony-I installation: modules Performed by Windows installer Performed by make;make install from source Use identical Slony-I versions on all servers! PostgreSQL servers may have different versions and run on different operating systems 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 28
  29. 29. Slony-I installation: Create cluster First node in cluster Node: database with installed cluster and running slon process Uses Slony-I creation scripts See pgAdmin's slony path option 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 29
  30. 30. Slony-I installation: Join cluster Create node and copy replication configuration from existing node Installs software in current database from existing cluster 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 30
  31. 31. Slony-I installation: paths Path: describes how a slon process connects to other nodes Libpq connect string 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 31
  32. 32. Slony-I installation: listens Listen: instructs a node to poll events from other nodes 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 32
  33. 33. Slony-I cluster status See node statistics 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 33
  34. 34. Slony-I replication sets Set: collection of tables and sequences All table and sequence data originating on one node 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 34
  35. 35. Slony-I Tables Table needs unique index, PK preferred pgAdmin doesn‘t offer tables without unique index Select triggers on the table that Slony-I should disable on slave nodes 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 35
  36. 36. Slony-I subscriptions Table and sequences must be present in slave node before subscribing! Subscribed sets can‘t be modified; use merge set instead. 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 36
  37. 37. Slony-I DDL script replication Use replication to execute changes to subscribed tables to insure master and slave have identical definitions May replicate any DDL script 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 37
  38. 38. Slony-I switch over Gracefully exchange master and slave role to a set between two nodes Both nodes must be fully functional Function "move set" 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 38
  39. 39. Slony-I fail over Master node has failed Failover tries to restore as much data from slave nodes as possible Designate a new master out of the previous slaves Not yet supported in pgAdmin III V1.4 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 39
  40. 40. pgAdmin future Coming in V1.6: Slony-I failover support Set creation wizard Health analysis improvements 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 40
  41. 41. Slony-I and pgAdmin Conclusion Most Slony-I functions accessible through easy-to-use GUI At-a-glance view on cluster health Integrated with other administrative tasks PostgreSQL has integrated replication! 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 41

×