PostgreSQL Disaster Recovery with Barman

7,830 views
7,433 views

Published on

Published in: Technology
2 Comments
11 Likes
Statistics
Notes
  • @organiser1 This presentation is about PostgreSQL database recovery, not MS SQL Server
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Try SQL database recovery software that is specially designed for this purposes. You can trial version and they apparently show you the retrieved database.
    Supported SQL Server Versions
    MS SQL Server 2000, 2005, 2008, 2008 r2 and 2012

    For more information and free download click http://sqldatabaserecoverytool.blogspot.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
7,830
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
136
Comments
2
Likes
11
Embeds 0
No embeds

No notes for slide

PostgreSQL Disaster Recovery with Barman

  1. 1. GABRIELE BARTOLINIPGDay Australia 2013 - Melbourne, 4 February 2013
  2. 2. GABRIELE BARTOLINI• Co-Founder and Manager of 2ndQuadrant Italia • Data Architect, Business critical environments • Data warehousing• Co-Founder Italian PostgreSQL Users Group• Co-Founder PostgreSQL Europe• PostgreSQL Contributor and Advocate
  3. 3. DISCLAIMERThis talk assumes you are familiar with disaster recovery concepts and PostgreSQL implementation of Point In Time Recovery
  4. 4. BE AWAREIn 2ndQuadrant, all these concepts usually fit in a2 day workshop on Disaster Recovery and a 1 day workshop on Barman alone
  5. 5. OUTLINE• Business continuity / Disaster recovery for databases• Disaster recovery with Barman for PostgreSQL
  6. 6. PART IBusiness continuity / Disaster recovery for databases
  7. 7. BUSINESS CONTINUITYactivity performed by an organization to ensure that criticalbusiness functions will be available to customers, suppliers, regulators, and other entities that must have access to those functions - Wikipedia
  8. 8. INFORMATION TECHNOLOGY• Business continuity • High availability • Disaster recovery
  9. 9. LAW REQUIREMENTSIn Italy, the “Codice dell’Amministrazione Digitale”defines business continuity requirements for public administrations
  10. 10. DISASTER (touch wood)system/hardware failures unintentional errors natural disaster
  11. 11. REACT TO A DISASTERRecover systems, data and infrastructures
  12. 12. TOO LATE!Do not wait for a disaster to happen
  13. 13. PLAN FOR DISASTERS “Disasters” will happen. Be prepared.
  14. 14. “Plans are worthless, but planning is everything. There is a very great distinction because when you are planning for an emergencyyou must start with this one thing: the very definition of "emergency" is that it is unexpected, therefore it is not going to happen the way you are planning.” - Dwight D. Eisenhower
  15. 15. REGULAR CRASH TESTS
  16. 16. DATABASEDISASTER RECOVERY Let’s just focus on databases!
  17. 17. REQUIREMENTS• Automated backups • Retention policies• Notifications (anomalies) • Data protection• Frequency of backups • Availability for recovery
  18. 18. POSTGRES BACKUP• Hot backup • Physical Backup • MVCC • Full backup (base backup)• Logical Backup • Differential backup (WAL) • pg_dump
  19. 19. TRADITIONAL DR WITH POSTGRESQL• PostgreSQLprimitives for • Custom scripts DR are robust and reliable • Hard to integrate in:• High level skills • Backup solutions • DBA • Disaster Recovery plans • Sysadmins • Hard to test!
  20. 20. EXISTING TOOLS• Omni-PITR • WAL-E • WAL centric • EC2 centric, but ...• WALmgr • good • WAL centric • came later• pg-rman • Server centric
  21. 21. NONE FOR DRNone of them was a pure disaster recovery solution. We wanted something similar to Oracle’s RMAN.
  22. 22. FILLING A HOLEThe lack of a DR solution is a barrier towards the adoption of PostgreSQL from Oracle users’ point of view.
  23. 23. DESIDERATA• Hot, Full, Differential and • Archival and compression Incremental backups • WAL segments• Multiple servers • Periodical backups• Remote backup & recovery • Automation• Backup catalogues • Integration• Retention policies • Usability
  24. 24. WWW.PGBARMAN.ORG
  25. 25. PART IIDisaster recovery with Barman for PostgreSQL
  26. 26. BARMAN• GNU GPL 3 • Debian/Ubuntu package• Hosted on Sourceforge.net • Designed, developed, maintained by 2ndQuadrant• Python 2.6/2.7• PostgreSQL 8.4, 9.0, 9.1, 9.2• PyPI package• RPM package
  27. 27. Postgres Postgres Postgres Barman tape BarmanLAN, hybrid architecture centralised architecture
  28. 28. Postgres Continuous archiving SSH commands(WAL shipping via SSH) SQL commands Barman
  29. 29. Postgres Barman WAL secure channel cron Barman’s WAL archive
  30. 30. Postgres Barman Periodical backup (weekly) Full backup - Sat 1, 4AM Differential backup Backup catalogue Full backup - Sat 8, 4AM Full backup - Sat 15, 4AM Full backup - Sat 22, 4AM
  31. 31. WAL archive Backup 1 Backup 2 (100MB) (105MB)WAL WAL WAL WAL WAL WAL WAL WAL WAL WALSize: 100MB + 80MB == 260MB Size: 105MB + 80MB = 185MB 160MB 180MB
  32. 32. CONFIGURATION FILE[barman]barman_home = /srv/barmanbarman_user = barmanlog_file = /var/log/barman/barman.loglog_level = NOTICEcompression = gzip[production]description = Production PostgreSQLssh_command = ssh pg.2ndQuadrant.itconninfo = host=pg.2ndQuadrant.it user=postgrescompression = bzip2
  33. 33. MULTI-SERVER CONFIGURATION[barman]; General configuration; …[server_one]; Configuration for Server 1; …[server_two]; Configuration for Server 2; …[server_X]; …
  34. 34. MULTIPLE FILES INCLUSION[barman]; General configuration; …configuration_files_directory = /etc/barman.d
  35. 35. CONVENTION OVER CONFIGURATION global/per server options default directory layout
  36. 36. CONVENTIONAL DIRECTORIES FOR BARMAN• barman_home (/srv/barman) • server directory (/srv/barman/production) • base directory (/srv/barman/production/base) • WAL directory (/srv/barman/production/wals) • incoming directory (/srv/barman/production/incoming)
  37. 37. GLOBAL COMMANDS• List of managed servers • barman list-server• Maintenance operations • barman cron
  38. 38. SERVER COMMANDS• Information and • Backup control diagnostics • Recovery control • barman status • barman check • barman show-server • barman list-backup
  39. 39. BACKUP CONTROL• barman backup• barman show-backup• barman list-files • standalone, data, wal, full• barman delete
  40. 40. SHOW BACKUP• General • WAL • Servername, Postgres • Number of associated files version, status, ... • disk usage• Base backup • Context • Start/End time, first/last WAL, disk usage, ... • Previous/Next backup
  41. 41. RECOVERY CONTROL• Recovery target (full / point in time)• Local recovery • barman recover• Remote recovery • barman recover --remote-ssh-command
  42. 42. ADVANCED RECOVERY• Point In Time Recovery • --target-time = TIME • --target-xid = XID• Relocation of tablespaces • --tablespace NAME:LOCATION [...]
  43. 43. COMMON USE CASES• Accidental errors recovery• Disaster recovery• Sandbox server (BI, staging, ...)
  44. 44. RETENTION POLICIES• User-defined policy• Determines how long backups and related WAL files need to be retained for recovery procedures (point of recoverability): • basedon number of backups (REDUNDANCY) or time (RECOVERY WINDOW)• Currently implemented only in auto mode • through ‘barman cron’
  45. 45. RETENTION POLICY CONFIGURATION; Base backup retention policyretention_policy = redundancy 3retention_policy = recovery window of 3 months; WAL retention policywal_retention_policy = base; The following is currently not supportedwal_retention_policy = recovery window of 14 days
  46. 46. SPONSOR A FEATURE!• Sandbox recovery • Replication protocol support (client / server)• Export/Import • Incremental backup• Logical backup integration • reduce backup time and • pg_dump on sandbox size instances
  47. 47. OUR COMMITMENT• Keep it open source• Reinvest money from sale of DR turnkey solutions in R&D• Support and maintain RPM/Debian packages• Accept sponsorships for new features development
  48. 48. CSI PIEMONTE (One of the top 10 ICT companies in Italy for revenue) “We found in Barman the optimal solution for physical backup and disaster recovery of PostgreSQL databases. Barman isrobust and easy to use. Its command interface allows an easy integration with the existing management tools in our enviroment.” Sponsors of RPM package and WAL compression
  49. 49. CONCLUSIONS• Hides complexity of PITR / Keeps unaltered PITR strenghts• Not invasive• Fosters migrations from Oracle• “Standard de facto” for PostgreSQL Disaster Recovery• Advice: plan for DR (if you have not done it yet)
  50. 50. QUESTIONS?Gabriele.Bartolini@2ndQuadrant.it Twitter: _GBartolini_ www.pgbarman.org
  51. 51. THANK YOU!
  52. 52. LICENSEThis work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 3.0 Unported License Copyright (c) 2012, 2013 - 2ndQuadrant.it

×