Agnostic Database
        Replication...
             ...with Binary Logs




www.mor.ph
Methodology
    • Constant Binary Log Replication
      • Phase I - Logging
      • Phase II - Archiving
      • Phase III...
Methodology
      • Agnosticism?
        • The methodology described herein is not
             bound to one vendor’s DB

...
Phase I - Logging




www.mor.ph
Phase I - Logging


    • Set your DB server to create binary
      logs

    • Make logs a manageable size (<50MB)
    • ...
Phase I - Logging
               PostgreSQL Example

    ‣ postgresql.conf
      # allows archiving to be done
      archi...
Phase I - Logging
                    MySQL Example

    ‣ my.cnf
      # Where to store your binary logs
      log-bin=/v...
Phase II - Archiving




www.mor.ph
Phase II - Archiving

    • Ship logs to a reliable location
    • Ship logs regularly & often
    • Ensure file integrity
...
Phase III - Recovery




www.mor.ph
Phase III - Recovery

    • Ship logs from your backup server
    • Ensure log ordering
    • Constant incremental recover...
Phase III - Recovery
              PostgreSQL Example

    ‣ recovery.conf
        restore_command = '/var/pgsql/bin/warm-...
Phase III - Recovery
                  MySQL Example

    ‣ No built-in mechanism
        sudo -u mysql ./latest-dump-s3.p...
Failover - When disaster
            strikes!
    • “When” NOT “If”
      • Always assume your DB will go down
      • Hav...
Failover - When disaster
            strikes!

      • Ensure final log segment recovery
      • Make your standby DB ‘acti...
Warm Standby vs.
            Clustering

    • When to Cluster
    • Draw backs to clustering
    • The search for the hol...
Additional Resources
    • http://www.postgresql.org/docs/8.3/
      interactive/wal.html

    • http://dev.mysql.com/doc/...
Upcoming SlideShare
Loading in...5
×

Agnostic DB Replication With Binary Logs

9,345

Published on

Database replication can be quite demanding. It can take up a great deal of time and money to implement, even with a simple application. What's more, if you are a startup, you might not have a lot of resources to start with.

When you overlook replication and backup, unplanned events such as a downtime can cripple your Web application. Or worse yet, have significant negative impact on your overall business.

In this presentation, we'll discuss how to overcome the challenges of delivering reliable Web applications.

For info about the recorded webinar, please contact us.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,345
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
35
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Agnostic DB Replication With Binary Logs

  1. 1. Agnostic Database Replication... ...with Binary Logs www.mor.ph
  2. 2. Methodology • Constant Binary Log Replication • Phase I - Logging • Phase II - Archiving • Phase III - Recovery • Failover - What to do when disaster strikes! www.mor.ph
  3. 3. Methodology • Agnosticism? • The methodology described herein is not bound to one vendor’s DB • With proper scripting your code should be reusable for many vendors DB solutions • Non-Invasive • Drastically reduced downtime compared to traditional restore from backup methods www.mor.ph
  4. 4. Phase I - Logging www.mor.ph
  5. 5. Phase I - Logging • Set your DB server to create binary logs • Make logs a manageable size (<50MB) • To timeout or not to timeout? • Ensure time based sort-ability www.mor.ph
  6. 6. Phase I - Logging PostgreSQL Example ‣ postgresql.conf # allows archiving to be done archive_mode = on # command to use to archive a logfile segment archive_command = '/path/to/your/archiving/script %p' # force a logfile segment switch after this time archive_timeout = 60 ‣ Log naming 000000010000000000000000 000000010000000000000001 ... 00000001000000000000000f www.mor.ph
  7. 7. Phase I - Logging MySQL Example ‣ my.cnf # Where to store your binary logs log-bin=/var/log/mysql/bin/mysql-bin-log #Max size for each log segment max_binlog_size = 50MB ‣ Log naming mysql-bin-log.0001 mysql-bin-log.0002 ... www.mor.ph
  8. 8. Phase II - Archiving www.mor.ph
  9. 9. Phase II - Archiving • Ship logs to a reliable location • Ship logs regularly & often • Ensure file integrity • Do regular full dumps www.mor.ph
  10. 10. Phase III - Recovery www.mor.ph
  11. 11. Phase III - Recovery • Ship logs from your backup server • Ensure log ordering • Constant incremental recovery • Incremental Recovery is Replication www.mor.ph
  12. 12. Phase III - Recovery PostgreSQL Example ‣ recovery.conf restore_command = '/var/pgsql/bin/warm-standby-s3 /var/pgsql/tripfile /var/pgsql/backups/WAL/%f quot;%pquot;' www.mor.ph
  13. 13. Phase III - Recovery MySQL Example ‣ No built-in mechanism sudo -u mysql ./latest-dump-s3.pl | mysql -h localhost www.mor.ph
  14. 14. Failover - When disaster strikes! • “When” NOT “If” • Always assume your DB will go down • Have a failover plan A • Have a failover plan B • Best laid plan will fall to waste if you do not test/validate regularly www.mor.ph
  15. 15. Failover - When disaster strikes! • Ensure final log segment recovery • Make your standby DB ‘active’ • Update client apps • Start the replication chain again! www.mor.ph
  16. 16. Warm Standby vs. Clustering • When to Cluster • Draw backs to clustering • The search for the holy 100% transparent clustering... www.mor.ph
  17. 17. Additional Resources • http://www.postgresql.org/docs/8.3/ interactive/wal.html • http://dev.mysql.com/doc/refman/6.0/en/ backup-and-recovery.html • http://dev.mysql.com/doc/refman/6.0/en/ point-in-time-recovery.html • http://dev.mysql.com/doc/refman/6.0/en/ backup.html www.mor.ph
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×