Replicating PostgreSQL Databases Using Slony-I

  • 4,512 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,512
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
154
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Slony-I Replication for Asynchronous PostgreSQL Slony-I on Microsoft Windows Dave Page 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 1
  • 2. Slony-I Asynchronous Replication for PostgreSQL Developed by Jan Wieck of Afilias Now a community project 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 2
  • 3. Uses Load balancing Redundancy/Failover Remote/distributed servers Upgrades 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 3
  • 4. Architecture 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 4
  • 5. Flexible Topology 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 5
  • 6. Object types Data Sequence values Schema changes 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 6
  • 7. Porting team Hiroshi Saito pgAdmin, psqlODBC, Npgsql, pgInstaller Dave Page pgAdmin, PostgreSQL, psqlODBC, pgInstaller, Npgsql, pgWeb Magnus Hagander PostgreSQL Server, pgInstaller, pgWeb Andreas Pflug pgAdmin, PostgreSQL Server 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 7
  • 8. Hiroshi Provided the initial ‘quick n dirty’ port. Organised the project 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 8
  • 9. Dave ‘Ducttape’ test suite New regression test suite Build system/Makefiles Patch/CVS management 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 9
  • 10. Magnus Code port slonik – based on Hiroshi’s work slon Reuse pgpipe from PostgreSQL Service control code Event logging 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 10
  • 11. Andreas GUI Management using pgAdmin 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 11
  • 12. Slony-I Replication for Asynchronous PostgreSQL Porting Slony-I Magnus Hagander 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 12
  • 13. Porting overview The bad Designed for Unix Relied on Unix tools and architecture The good Portable between Unixes Based on PostgreSQL build system 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 13
  • 14. Porting Slonik - easy No shell utils available Slonik 1.1 uses SED Hiroshi already fixed Path issues Find the ”share” directory Windows compatible paths 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 14
  • 15. Porting Slon - easy Pthreads Find library to link with Winsock Simple initialization issue Pipes Steal from PostgreSQL Signals Ignore! 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 15
  • 16. Porting slon – a bit more work Eventlog integration Centralised logging already Eventlog when service, stdout when console Create message library Versioning metadata Steal most from PostgreSQL Decimal version number in config.h 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 16
  • 17. Porting slon – most work fork() Service integration Two problems, one solution 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 17
  • 18. Slon – Unix architecture init rc.slony rc.xyz slon watchdog slon watchdog fork() fork() slon engine slon engine 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 18
  • 19. Slon – Windows architecture Service Control Manager init rc.slony rc.xyz slon slon win32 service handler watchdog slon watchdog fork() CreateProcess() CreateProcess() fork() slon engine slon engine 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 19
  • 20. Porting – end result Slon runs on the commandline Only for testing/debugging! Single service, multiple engines One config file per engine Paths stored in registry, add/remove with slon commandline Multiple services, multiple engines Different versions of slon 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 20
  • 21. Simple slony replication DEMO 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 21
  • 22. Base table creation db1 CREATE TABLE t ( name text NOT NULL PRIMARY KEY) INSERT INTO t VALUES (’Dave’) INSERT INTO t VALUES (’Magnus’) db2 CREATE TABLE t ( name text NOT NULL PRIMARY KEY) 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 22
  • 23. Installing Slony slon –regservice slon –addengine c:slonydb1.conf slon –addengine c:slonydb2.conf slon -listengines db1.conf log_level=1 log_timestamp=false cluster_name='test' conn_info='host=127.0.0.1 user=postgres dbname=db1' 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 23
  • 24. Slon setup script # Create slony cluster cluster name = test; node 1 admin conninfo = ’host=127.0.0.1 user=postgres database=db1’; node 2 admin conninfo = ’host=127.0.0.1 user=postgres database=db2’; init cluster (id=1, comment=’Node 1’) # Create set of tables with one table create set (id=1, origin=1) set add table (set id=1, origin=1, id=1, fully qualified name = ’public.t’) 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 24
  • 25. Slon setup script (contd) # Create node for second engine store node (id=2, comment=’Node 2’); # Create paths between the two nodes store path (server=1,client=2, conninfo=’host=127.0.0.1 user=postgres dbname=db1’); store path (server=2,client=1, conninfo=’host=127.0.0.1 user=postgres dbname=db2’); store listen (origin=1, provider=1, receiver=2); store listen (origin=2, provider=2, receiver=1); # Subscribe the slave subscribe set (id=1, provider=1, receiver=2, forward=no) 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 25
  • 26. Slony-I Replication for Asynchronous PostgreSQL Graphical management of Slony-I Andreas Pflug 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 26
  • 27. pgAdmin III architecture C++ wxWidgets 2.6 Native libpq PostgreSQL connection Native Windows and GTK2 look and feel For PostgreSQL 7.3 and above Some helper programs and modules 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 27
  • 28. Slony-I installation: modules Performed by Windows installer Performed by make;make install from source Use identical Slony-I versions on all servers! PostgreSQL servers may have different versions and run on different operating systems 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 28
  • 29. Slony-I installation: Create cluster First node in cluster Node: database with installed cluster and running slon process Uses Slony-I creation scripts See pgAdmin's slony path option 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 29
  • 30. Slony-I installation: Join cluster Create node and copy replication configuration from existing node Installs software in current database from existing cluster 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 30
  • 31. Slony-I installation: paths Path: describes how a slon process connects to other nodes Libpq connect string 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 31
  • 32. Slony-I installation: listens Listen: instructs a node to poll events from other nodes 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 32
  • 33. Slony-I cluster status See node statistics 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 33
  • 34. Slony-I replication sets Set: collection of tables and sequences All table and sequence data originating on one node 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 34
  • 35. Slony-I Tables Table needs unique index, PK preferred pgAdmin doesn‘t offer tables without unique index Select triggers on the table that Slony-I should disable on slave nodes 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 35
  • 36. Slony-I subscriptions Table and sequences must be present in slave node before subscribing! Subscribed sets can‘t be modified; use merge set instead. 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 36
  • 37. Slony-I DDL script replication Use replication to execute changes to subscribed tables to insure master and slave have identical definitions May replicate any DDL script 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 37
  • 38. Slony-I switch over Gracefully exchange master and slave role to a set between two nodes Both nodes must be fully functional Function "move set" 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 38
  • 39. Slony-I fail over Master node has failed Failover tries to restore as much data from slave nodes as possible Designate a new master out of the previous slaves Not yet supported in pgAdmin III V1.4 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 39
  • 40. pgAdmin future Coming in V1.6: Slony-I failover support Set creation wizard Health analysis improvements 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 40
  • 41. Slony-I and pgAdmin Conclusion Most Slony-I functions accessible through easy-to-use GUI At-a-glance view on cluster health Integrated with other administrative tasks PostgreSQL has integrated replication! 28th October 2005 Dave Page - Magnus Hagander - Andreas Pflug 41