Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

Advanced backup methods (Postgres@CERN) Slide 1 Advanced backup methods (Postgres@CERN) Slide 2 Advanced backup methods (Postgres@CERN) Slide 3 Advanced backup methods (Postgres@CERN) Slide 4 Advanced backup methods (Postgres@CERN) Slide 5 Advanced backup methods (Postgres@CERN) Slide 6 Advanced backup methods (Postgres@CERN) Slide 7 Advanced backup methods (Postgres@CERN) Slide 8 Advanced backup methods (Postgres@CERN) Slide 9 Advanced backup methods (Postgres@CERN) Slide 10 Advanced backup methods (Postgres@CERN) Slide 11 Advanced backup methods (Postgres@CERN) Slide 12 Advanced backup methods (Postgres@CERN) Slide 13 Advanced backup methods (Postgres@CERN) Slide 14 Advanced backup methods (Postgres@CERN) Slide 15 Advanced backup methods (Postgres@CERN) Slide 16 Advanced backup methods (Postgres@CERN) Slide 17 Advanced backup methods (Postgres@CERN) Slide 18 Advanced backup methods (Postgres@CERN) Slide 19 Advanced backup methods (Postgres@CERN) Slide 20 Advanced backup methods (Postgres@CERN) Slide 21 Advanced backup methods (Postgres@CERN) Slide 22 Advanced backup methods (Postgres@CERN) Slide 23 Advanced backup methods (Postgres@CERN) Slide 24 Advanced backup methods (Postgres@CERN) Slide 25 Advanced backup methods (Postgres@CERN) Slide 26 Advanced backup methods (Postgres@CERN) Slide 27 Advanced backup methods (Postgres@CERN) Slide 28 Advanced backup methods (Postgres@CERN) Slide 29 Advanced backup methods (Postgres@CERN) Slide 30 Advanced backup methods (Postgres@CERN) Slide 31 Advanced backup methods (Postgres@CERN) Slide 32 Advanced backup methods (Postgres@CERN) Slide 33 Advanced backup methods (Postgres@CERN) Slide 34 Advanced backup methods (Postgres@CERN) Slide 35 Advanced backup methods (Postgres@CERN) Slide 36 Advanced backup methods (Postgres@CERN) Slide 37 Advanced backup methods (Postgres@CERN) Slide 38 Advanced backup methods (Postgres@CERN) Slide 39 Advanced backup methods (Postgres@CERN) Slide 40 Advanced backup methods (Postgres@CERN) Slide 41 Advanced backup methods (Postgres@CERN) Slide 42 Advanced backup methods (Postgres@CERN) Slide 43 Advanced backup methods (Postgres@CERN) Slide 44 Advanced backup methods (Postgres@CERN) Slide 45 Advanced backup methods (Postgres@CERN) Slide 46 Advanced backup methods (Postgres@CERN) Slide 47
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

1 Like

Share

Download to read offline

Advanced backup methods (Postgres@CERN)

Download to read offline

This talk cover various advanced topics in the area of backups:
- incremental backups;
- archive management;
- backup validation;
- retention policies;
etc.
Based on these features, we'll compare various backup/recovery solutions for PostgreSQL.
This information will help you to choose the most appropriate tool for your system.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Advanced backup methods (Postgres@CERN)

  1. 1. Advanced PostgreSQL backup & recovery methods Anastasia Lubennikova Postgres@CERN 2020 1
  2. 2. Agenda - Why backup? - What is a good backup tool? - Overview of advanced backup features - Overview of PostgreSQL backup tools Spoiler: this talk doesn’t contain any benchmarks. 2
  3. 3. Why do you need a backup? - To restore the database after an accident - hardware failure - software bug - human error - To set up a new replica - To create a test environment - To inspect data from the past 3
  4. 4. What are the options? - replica is not a backup - dump a.k.a. “logical backup” - storage snapshots - pg_basebackup - set of custom scripts - PostgreSQL specific backup tools 4
  5. 5. What makes a good backup tool? - Convenience - out-of-box automatization of various routines - documentation & support - convenient and stable api - Performance - parallel execution - compression - incremental & differential backups - WAL prefetch 5
  6. 6. What backup tools exist? - Barman - pgBackRest - pg_probackup - WAL-G - BART - part of the “EDB Advanced Server” - requires pg_basebackup 6
  7. 7. Who is who? Barman - https://www.pgbarman.org/ - 2ndQuadrant - GPL v 3.0 - Python - first release: 2011 - Two methods: basebackup & rsync Notable features: Synchronous streaming for “zero data loss”. 7
  8. 8. Who is who? pgBackRest - https://pgbackrest.org/ - Crunchy Data - MIT License - C - first release: 2014 Notable features: Performance optimizations for large backups. 8
  9. 9. Who is who? pg_probackup - https://github.com/postgrespro/pg_probackup - Postgres Professional - PostgreSQL License - C - first release: 2017 (based on pg_arman) Notable features: Page-level incremental backups and built-in validation. 9
  10. 10. Who is who? WAL-G - https://github.com/wal-g/wal-g - introduced by Citus Data, now maintained by Yandex Cloud team - Apache License, Version 2.0 - Go - first release: 2017 ( “based on” WAL-E) Notable features: Out-of-box support for various cloud storages. 10
  11. 11. Feature list 1. Documentation & Support 2. Backup management 3. WAL archive management 4. Incremental backups 5. Compression and parallel execution 6. Remote backup 7. Cloud backup 8. Advanced restore options 9. Backup validation 10. Backup retention 11
  12. 12. 1. Documentation & Support 12
  13. 13. Documentation Barman User guide & command reference. Great overview of backup architectures pgBackRest User guide & command reference pg_probackup User guide & command reference WAL-G README 13
  14. 14. Installation Barman Linux packages, Build from source pgBackRest Linux packages, Build from source pg_probackup Linux packages, Build from source, Windows installer WAL-G Linux binary, Build from source 14
  15. 15. Support: bug fixes Barman https://github.com/2ndquadrant-it/barman/issues pgBackRest https://github.com/pgbackrest/pgbackrest/issues pg_probackup https://github.com/postgrespro/pg_probackup/issues WAL-G https://github.com/wal-g/wal-g/issues 15
  16. 16. Commercial support Barman 2ndQuadrant pgBackRest CrunchyData pg_probackup Postgres Professional WAL-G 16
  17. 17. 2. Backup management 17
  18. 18. Set up new PostgreSQL instance Barman server configuration files pgBackRest stanza configuration files pg_probackup instance configuration files, set-config command WAL-G - config via environment variables 18
  19. 19. Backup information Barman plain pgBackRest plain, json + postgresql table pg_probackup plain, json + detailed wal archive info WAL-G plain, json 19
  20. 20. 3. WAL archive management 20
  21. 21. WAL archive management Barman rsync / get-wal pgBackRest archive-push / archive get archive-async pg_probackup archive-push / archive-get WAL-G wal-push / wal-fetch wal prefetch 21
  22. 22. Streaming backups - Recovery Point Objective (RPO): "maximum targeted period in which data might be lost from an IT service due to a major incident" - “RPO = 0” (Zero data loss) can be achieved by synchronous WAL streaming - replication slot prevents the removal of WAL that is not yet received (PostgreSQL feature) 22
  23. 23. Streaming backups Barman streaming_archiver (pg_recievewal) replication slot pgBackRest pg_probackup backup --stream replication slot WAL-G 23
  24. 24. 4. Incremental backups Full backup includes all data files. Differential backup contains changes since last full backup. Incremental backup contains changes since last backup. 24
  25. 25. Incremental backup methods - DELTA - read everything, backup what changed - independent method - read load on data server - PAGE - scan WAL to determine changed blocks - requires WAL archive - minimal load on data server - PTRACK - remember changed blocks in a map - requires core patch - minimal load during backup 25
  26. 26. Incremental backups Barman file-level incremental (DELTA) pgBackRest file-level incremental (DELTA) file-level differential (DELTA) pg_probackup page-level incremental: DELTA, PAGE, PTRACK WAL-G page-level incremental (DELTA) 26
  27. 27. 5. Compression and parallel execution 27
  28. 28. 6. Remote backup Barman SSH pgBackRest SSH pg_probackup SSH WAL-G 28
  29. 29. 7. Cloud backup 29
  30. 30. Backup to cloud storage Barman scripts to ship backups to S3 pgBackRest Amazon S3 + encryption pg_probackup WAL-G Amazon S3, Google Cloud Storage, Azure Storage, Swift Object Storage + encryption 30
  31. 31. Extra backup features - Backup from standby (All tools) - to reduce load on master data server - Resume backup (only pgBackRest) 31
  32. 32. 8. Advanced restore options. PITR Restore to a certain moment in time. 32
  33. 33. Point-in-time-recovery Barman recovery target options pgBackRest recovery target options pg_probackup recovery target options WAL-G 33
  34. 34. Partial restore Barman pgBackRest restore selected databases pg_probackup restore selected databases WAL-G 34
  35. 35. 9. Backup validation 35
  36. 36. Validate backups Barman DIY with custom hooks on backup & restore pgBackRest page checksums on backup pg_probackup page checksums on backup validate on demand check instance WAL-G 36
  37. 37. 10. Backup retention 37
  38. 38. 10. Backup retention. Redundancy = 3 38
  39. 39. 10. Backup retention. Window = 7 days 39
  40. 40. Retention policy Barman retention_policy = REDUNDANCY retention_policy = RECOVERY WINDOW pgBackRest redundancy pg_probackup --retention-redundancy --retention-window WAL-G redundancy: retain N window: delete before 40
  41. 41. Backup pinning Barman pgBackRest pg_probackup ttl=0 WAL-G backup-mark 41
  42. 42. Archive retention Barman pgBackRest Archive Retention --repo-retention-archive pg_probackup delete --expired --wal --wal-depth=1 WAL-G 42
  43. 43. Backup merging Save space by merging old incremental backups. 43
  44. 44. Backup merging Barman pgBackRest pg_probackup merge --merge-expired WAL-G 44
  45. 45. 45
  46. 46. Conclusion Barman (rsync) pgBackRest pg_probackup WAL-G Support + + + + Backup management + + + - WAL management + + + + Incremental backup + + + + Compression & parallel execution + + + + 46
  47. 47. Conclusion Barman (rsync) pgBackRest pg_probackup WAL-G Remote backup + + + + Cloud backup - + - + Advanced restore + + + - Backup validation + + + - Backup retention + + + + 47
  • AiratMurde

    Jan. 21, 2020

This talk cover various advanced topics in the area of backups: - incremental backups; - archive management; - backup validation; - retention policies; etc. Based on these features, we'll compare various backup/recovery solutions for PostgreSQL. This information will help you to choose the most appropriate tool for your system.

Views

Total views

1,515

On Slideshare

0

From embeds

0

Number of embeds

30

Actions

Downloads

49

Shares

0

Comments

0

Likes

1

×