Your SlideShare is downloading. ×
PostgreSQL Replication <ul><li>Joshua D. Drake / Command Prompt, Inc. </li></ul><ul><li>Major Contributor PostgreSQL.Org <...
Why <ul><li>PostgreSQL + Replicator represents the easiest and most flexible PostgreSQL replication solution to date. </li...
What features does it support? <ul><li>Master to many (Asynchronous) slaves </li></ul><ul><li>Partial Sets </li></ul><ul><...
Open Source + Supported <ul><li>BSD Licensed </li></ul><ul><li>Open Community </li></ul><ul><ul><li>https://projects.comma...
Get it! <ul><li>RPM: </li></ul><ul><ul><li>http://www.pgsqlrpms.org/ </li></ul></ul><ul><li>Source: </li></ul><ul><ul><li>...
Compile it <ul><li>jd@hardy:~/repos$ cd REPLICATOR-8.3-1.8/ </li></ul><ul><li>jd@hardy:~/repos/REPLICATOR-8.3-1.8$ autocon...
Install it <ul><li>jd@hardy:~/repos/REPLICATOR-8.3-1.8$ make -j2 install </li></ul><ul><ul><li>make -C doc install </li></...
Test it <ul><li>jd@hardy:~/repos/REPLICATOR-8.3-1.8$ cd  src/test/replication </li></ul><ul><li>jd@hardy:~ PGROOT=/tmp/rep...
If only it could be... always <ul><li>test1 succeeded </li></ul><ul><li>test2 succeeded </li></ul><ul><li>test4 succeeded ...
Initialize Master <ul><li>jd@hardy:~/$ cd /tmp/replicator </li></ul><ul><li>jd@hardy:/$ bin/initdb -D data </li></ul><ul><...
The postgresql.conf <ul><li>replication_enable = false </li></ul><ul><li>replication_mode = master  </li></ul><ul><li>repl...
postgresql.conf (2) <ul><li>replication_mcp_require_ssl = false </li></ul><ul><li>replication_compression = true  </li></u...
Apply useful functions <ul><li>pg_ctl -D data start </li></ul><ul><li>psql -U jd repl_test  < /tmp/replicator/share/postgr...
The functions <ul><li>is_replicated(REGCLASS) </li></ul><ul><li>get_replication_status(REGCLASS) </li></ul><ul><li>enable_...
The MCP <ul><li>Master Control Process </li></ul><ul><ul><li>Manages replication between nodes </li></ul></ul><ul><ul><ul>...
mcp_server.conf <ul><li>mcp_listen_address  = &quot;0.0.0.0&quot; </li></ul><ul><li>mcp_listen_port  = 7777 </li></ul><ul>...
mcp_server.conf (2) <ul><li>mcp_queue_min_size  = 16000000 </li></ul><ul><li>mcp_queue_max_size  = 32000000 </li></ul><ul>...
Fire up <ul><li>Start the mcp </li></ul><ul><ul><li>mcp_server -D /data/mcp_data </li></ul></ul><ul><li>Start Replication ...
Check the status <ul><li>type  brecno  frecno  vrecno  lrecno  sync  connected  timestamp </li></ul><ul><li>master  888  8...
Fail over <ul><li>Two modes: </li></ul><ul><ul><li>PROMOTE </li></ul></ul><ul><ul><li>PROMOTE FORCE;  </li></ul></ul>
Upcoming SlideShare
Loading in...5
×

Go Replicator

666

Published on

A talk on understanding and configuring PostgreSQL Replicator

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
666
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Go Replicator"

  1. 1. PostgreSQL Replication <ul><li>Joshua D. Drake / Command Prompt, Inc. </li></ul><ul><li>Major Contributor PostgreSQL.Org </li></ul><ul><li>President PgUS </li></ul>
  2. 2. Why <ul><li>PostgreSQL + Replicator represents the easiest and most flexible PostgreSQL replication solution to date. </li></ul>
  3. 3. What features does it support? <ul><li>Master to many (Asynchronous) slaves </li></ul><ul><li>Partial Sets </li></ul><ul><li>Unique features </li></ul><ul><ul><li>Role replication </li></ul></ul><ul><ul><li>Grant replication </li></ul></ul><ul><ul><li>Large object replication </li></ul></ul><ul><ul><li>Transaction logs, not triggers </li></ul></ul><ul><li>Failover/Promote </li></ul><ul><li>Integrated, no triggers, easy DDL </li></ul>
  4. 4. Open Source + Supported <ul><li>BSD Licensed </li></ul><ul><li>Open Community </li></ul><ul><ul><li>https://projects.commandprompt.com/public/replicator </li></ul></ul><ul><li>Supported </li></ul><ul><ul><li>Commercially from http://www.commandprompt.com </li></ul></ul>
  5. 5. Get it! <ul><li>RPM: </li></ul><ul><ul><li>http://www.pgsqlrpms.org/ </li></ul></ul><ul><li>Source: </li></ul><ul><ul><li>#! svn co https://projects.commandprompt.com/public/replicator/repo </li></ul></ul><ul><li>Debian/Ubuntu </li></ul><ul><ul><li>Working on it </li></ul></ul>
  6. 6. Compile it <ul><li>jd@hardy:~/repos$ cd REPLICATOR-8.3-1.8/ </li></ul><ul><li>jd@hardy:~/repos/REPLICATOR-8.3-1.8$ autoconf </li></ul><ul><li>jd@hardy:~/repos/REPLICATOR-8.3-1.8$ ./configure --prefix=/tmp/replicator </li></ul><ul><ul><li>checking build system type... i686-pc-linux-gnu </li></ul></ul><ul><ul><li>checking host system type... i686-pc-linux-gnu </li></ul></ul><ul><ul><li>checking which template to use... linux </li></ul></ul><ul><ul><li>checking whether to build with 64-bit integer date/time support... no </li></ul></ul><ul><ul><li>checking whether NLS is wanted... no </li></ul></ul><ul><ul><li>checking for default port number... 5432 </li></ul></ul><ul><ul><li>[...] </li></ul></ul>
  7. 7. Install it <ul><li>jd@hardy:~/repos/REPLICATOR-8.3-1.8$ make -j2 install </li></ul><ul><ul><li>make -C doc install </li></ul></ul><ul><ul><li>make[1]: Entering directory `/home/jd/repos/REPLICATOR-8.3-1.8/doc' </li></ul></ul><ul><ul><li>make[1]: Nothing to be done for `install'. </li></ul></ul><ul><ul><li>make[1]: Leaving directory `/home/jd/repos/REPLICATOR-8.3-1.8/doc' </li></ul></ul><ul><ul><li>make -C src install </li></ul></ul><ul><li>[...] </li></ul>
  8. 8. Test it <ul><li>jd@hardy:~/repos/REPLICATOR-8.3-1.8$ cd src/test/replication </li></ul><ul><li>jd@hardy:~ PGROOT=/tmp/replicator ./test.pl | grep -i succeeded </li></ul>
  9. 9. If only it could be... always <ul><li>test1 succeeded </li></ul><ul><li>test2 succeeded </li></ul><ul><li>test4 succeeded </li></ul><ul><li>test5 succeeded </li></ul><ul><li>test6 succeeded </li></ul><ul><li>test7 succeeded </li></ul><ul><li>test8 succeeded </li></ul><ul><li>test9 succeeded </li></ul><ul><li>test10 succeeded </li></ul><ul><li>test11 succeeded </li></ul><ul><li>test12 succeeded </li></ul><ul><li>test14 succeeded </li></ul><ul><li>test15 succeeded </li></ul><ul><li>test16 succeeded </li></ul><ul><li>test17 succeeded </li></ul><ul><li>test_lo succeeded </li></ul>
  10. 10. Initialize Master <ul><li>jd@hardy:~/$ cd /tmp/replicator </li></ul><ul><li>jd@hardy:/$ bin/initdb -D data </li></ul><ul><li>jd@hardy:/$ bin/pg_ctl -D data start </li></ul><ul><li>jd@hardy:/$ bin/createdb -U postgres repl_test; </li></ul><ul><li>jd@hardy:/$ bin/pg_ctl -D data stop </li></ul><ul><li>jd@hardy:/$ bin/init-mammoth-database -D data repl_test </li></ul><ul><li>Database repl_test ready for </li></ul><ul><li>Mammoth Replicator </li></ul>
  11. 11. The postgresql.conf <ul><li>replication_enable = false </li></ul><ul><li>replication_mode = master </li></ul><ul><li>replication_slave_no = 0 </li></ul><ul><li>replication_database = 'repl_test' </li></ul><ul><li>replication_mcp_address = '127.0.0.1' </li></ul><ul><li>replication_mcp_port = 7777 </li></ul><ul><li>replication_mcp_authkey = 'mcp-key' </li></ul><ul><li>replication_data_path = 'rlog' </li></ul>
  12. 12. postgresql.conf (2) <ul><li>replication_mcp_require_ssl = false </li></ul><ul><li>replication_compression = true </li></ul><ul><li>replication_use_utf8_encoding = true </li></ul>
  13. 13. Apply useful functions <ul><li>pg_ctl -D data start </li></ul><ul><li>psql -U jd repl_test < /tmp/replicator/share/postgresql/replicator-functions.sql </li></ul>
  14. 14. The functions <ul><li>is_replicated(REGCLASS) </li></ul><ul><li>get_replication_status(REGCLASS) </li></ul><ul><li>enable_replication(NAME, INTEGER[]) enable_replication_by_schema(NAME, INTEGER[]) </li></ul><ul><li>enable_replication_global(INTEGER[]) </li></ul><ul><li>disable_replication(REGCLASS) </li></ul><ul><li>disable_replication_by_schema(NAME) </li></ul><ul><li>disable_replication_global() RETURNS VOID AS $$ </li></ul>
  15. 15. The MCP <ul><li>Master Control Process </li></ul><ul><ul><li>Manages replication between nodes </li></ul></ul><ul><ul><ul><li>Transactions </li></ul></ul></ul><ul><ul><ul><li>Promotion </li></ul></ul></ul><ul><ul><ul><li>Status </li></ul></ul></ul><ul><ul><li>Has queueing mechanism </li></ul></ul><ul><ul><li>Unlimited (in theory) slaves without impact on master </li></ul></ul><ul><ul><li>Operates on any server </li></ul></ul>
  16. 16. mcp_server.conf <ul><li>mcp_listen_address = &quot;0.0.0.0&quot; </li></ul><ul><li>mcp_listen_port = 7777 </li></ul><ul><li>mcp_authkey = &quot;mcp-key&quot; </li></ul><ul><li>mcp_master_address = &quot;127.0.0.1&quot; </li></ul><ul><li>mcp_max_slaves = 5 </li></ul><ul><li>mcp_slave_addresses = &quot;0:192.168.0.1, 1:192.168.0.1&quot; </li></ul><ul><li>mcp_promote_allow = &quot;&quot; </li></ul>
  17. 17. mcp_server.conf (2) <ul><li>mcp_queue_min_size = 16000000 </li></ul><ul><li>mcp_queue_max_size = 32000000 </li></ul><ul><li>mcp_require_ssl = 0 </li></ul><ul><li>log_level = &quot;log&quot; </li></ul><ul><li>replication_data_path = &quot;rlog” </li></ul>
  18. 18. Fire up <ul><li>Start the mcp </li></ul><ul><ul><li>mcp_server -D /data/mcp_data </li></ul></ul><ul><li>Start Replication on relation </li></ul><ul><ul><li>Make sure you have a primary key </li></ul></ul><ul><ul><li>ALTER TABLE foo ENABLE REPLICATION </li></ul></ul><ul><ul><li>ALTER TABLE foo ENABLE REPLICATION ON SLAVE 0; </li></ul></ul><ul><li>Replicate the whole database </li></ul><ul><ul><li>SELECT enable_replication_global('{0}') </li></ul></ul>
  19. 19. Check the status <ul><li>type brecno frecno vrecno lrecno sync connected timestamp </li></ul><ul><li>master 888 888 0 497519 sync true Sat Jan 31 08:41:35 </li></ul><ul><li>slave 0 - 497520 497519 - sync true Sat Jan 31 08:41:35 </li></ul><ul><li>slave 1 - 888 0 - desync false Wed Dec 31 16:00:00 </li></ul><ul><li>slave 2 - 888 0 - desync false Wed Dec 31 16:00:00 </li></ul><ul><li>slave 3 - 888 0 - desync false Wed Dec 31 16:00:00 </li></ul><ul><li>slave 4 - 888 0 - desync false Wed Dec 31 16:00:00 </li></ul><ul><li>(In the normal output year is there too) </li></ul>
  20. 20. Fail over <ul><li>Two modes: </li></ul><ul><ul><li>PROMOTE </li></ul></ul><ul><ul><li>PROMOTE FORCE; </li></ul></ul>

×