Rapid Upgrades With Pg_Upgrade, Bruce Momjian

1,301 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,301
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Rapid Upgrades With Pg_Upgrade, Bruce Momjian

  1. 1. Rapid Upgrades With Pg_Upgrade BRUCE MOMJIAN, ENTERPRISEDB May, 2010 Abstract Pg_Upgrade allows migration between major releases of Postgres without a data dump/reload. This presentation explains how pg_upgrade works. Creative Commons Attribution License http://momjian.us/presentations
  2. 2. Traditional Postgres Upgrade Options   pg_dump (logical dump)/restore   Slony Rapid Upgrades With Pg-Upgrade 2
  3. 3. Why Upgrading Postgres Is Complex   New features often require system table changes   However, the internal data format rarely changes Rapid Upgrades With Pg-Upgrade 3
  4. 4. Why Pg_Upgrade   Very fast upgrades   Optionally no additional disk space pg_upgrade installs new system tables while using data files from the previous Postgres version. Rapid Upgrades With Pg-Upgrade 4
  5. 5. How It Works: Initial Setup 1 2 3 6 5 4 7 8 9 pg_class User Tables 15 21 27 262014 13 19 25 241812 17 2311 10 16 22 System Tables clog Old Cluster 1 2 3 6 5 4 7 8 9 pg_class User Tables System Tables clog New Cluster Rapid Upgrades With Pg-Upgrade 5
  6. 6. Decouple New Clog Via Freezing 1 2 3 6 5 4 7 8 9 pg_class User Tables 15 21 27 262014 13 19 25 241812 17 2311 10 16 22 System Tables clog Old Cluster 1 2 3 6 4 7 8 9 pg_class User Tables System Tables clog New Cluster 5 Freeze X X Rapid Upgrades With Pg-Upgrade 6
  7. 7. Transfer Clog and XID 1 2 3 6 5 4 7 8 9 pg_class User Tables 15 21 27 262014 13 19 25 241812 17 2311 10 16 22 System Tables clog Old Cluster 1 2 3 6 5 4 7 8 9 pg_class User Tables System Tables clog New Cluster controldata controldataxid Rapid Upgrades With Pg-Upgrade 7
  8. 8. Get Schema Dump 1 2 3 6 5 4 7 8 9 pg_class User Tables 15 21 27 262014 13 19 25 241812 17 2311 10 16 22 System Tables clog Old Cluster 1 2 3 6 5 4 7 8 9 pg_class User Tables System Tables clog New Cluster pg_dumpall - -schema Rapid Upgrades With Pg-Upgrade 8
  9. 9. Restore Schema In New Cluster 1 2 3 6 5 4 7 8 9 pg_class User Tables 15 21 27 262014 13 19 25 241812 17 2311 10 16 22 System Tables clog Old Cluster 1 2 3 6 5 4 7 8 9 pg_class User Tables 26 System Tables clog New Cluster pg_dumpall − −schema 17 23 24 25 27 18 10 11 12 13 14 15 16 19 20 21 22 Rapid Upgrades With Pg-Upgrade 9
  10. 10. Copy User Heap/Index Files 1 2 3 6 5 4 7 8 9 pg_class User Tables 15 21 27 262014 13 19 25 241812 17 2311 10 16 22 System Tables clog Old Cluster 1 2 3 6 5 4 7 8 9 pg_class User Tables 26 System Tables clog New Cluster 1610 11 13 12 14 15 19 20 21 22 23 24 25 27 18 17 Rapid Upgrades With Pg-Upgrade 10
  11. 11. Complete 1 2 3 6 5 4 7 8 9 pg_class User Tables 15 21 27 262014 13 19 25 241812 17 2311 10 16 22 System Tables clog Old Cluster 1 2 3 6 5 4 7 8 9 pg_class User Tables 26 System Tables clog New Cluster 24 25 27 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Rapid Upgrades With Pg-Upgrade 11
  12. 12. How It Works: In Detail   Check for cluster compatability – locale – encoding   Use pg_dumpall to dump old cluster schema (no data)   Freeze all new cluster rows (remove reference to clog entries)   New cluster uses old xid counter value (see freeze above) – Set system table frozen xids to match the current xid   Create new users/databases   Collect cluster informationRapid Upgrades With Pg-Upgrade 12
  13. 13.   Install support functions that call internal backend functions   Create schema in new cluster   Copy or link files from old cluster to new cluster   Warn about any remaining issues, like REINDEX requirements Rapid Upgrades With Pg-Upgrade 13
  14. 14. Sample Run: Performing Consistency Checks Performing Consistency Checks ----------------------------- Checking old data directory (/u/pgsql.old/data) ok Checking old bin directory (/u/pgsql.old/bin) ok Checking new data directory (/u/pgsql/data) ok Checking new bin directory (/u/pgsql/bin) ok Checking for /contrib/isn with bigint-passing mismatch ok Checking for large objects ok Creating catalog dump ok Checking for presence of required libraries ok | If pg_upgrade fails after this point, you must | re-initdb the new cluster before continuing. | You will also need to remove the ".old" suffix | from /u/pgsql.old/data/global/pg_control.old. Rapid Upgrades With Pg-Upgrade 14
  15. 15. Sample Run: Performing Migration Performing Migration -------------------- Adding ".old" suffix to old global/pg_control ok Analyzing all rows in the new cluster ok Freezing all rows on the new cluster ok Deleting new commit clogs ok Copying old commit clogs to new server ok Setting next transaction id for new cluster ok Resetting WAL archives ok Setting frozenxid counters in new cluster ok Creating databases in the new cluster ok Adding support functions to new cluster ok Restoring database schema to new cluster ok Removing support functions from new cluster ok Restoring user relation files ok Setting next oid for new cluster ok Creating script to delete old cluster ok Checking for large objects ok Rapid Upgrades With Pg-Upgrade 15
  16. 16. Sample Run: Completion Upgrade complete ---------------- | Optimizer statistics is not transferred by pg_upgrade | so consider running: | vacuumdb --all --analyze-only | on the newly-upgraded cluster. | Running this script will delete the old cluster’s data files: | /u/postgres/pg_upgrade_output/delete_old_cluster.sh Rapid Upgrades With Pg-Upgrade 16
  17. 17. Possible Data Format Changes Change Conversion Method clog none heap page header, including bitmask convert to new page format on read tuple header, including bitmask convert to new page format on read data value format create old data type in new cluster index page format reindex, or recreate index methods TOAST page format convert to new page format on read Rapid Upgrades With Pg-Upgrade 17
  18. 18. Migration Timings Migration Method Minutes dump/restore 300.0 dump with parallel restore 180.0 pg_upgrade in copy mode 44.0 pg_upgrade in link mode 0.7 Database size: 150GB, 850 tables The last duration is 44 seconds. Timings courtesy of Stefan Kaltenbrunner (mastermind on IRC) Rapid Upgrades With Pg-Upgrade 18
  19. 19. PostgreSQL Version Compatibility New Old 8.3 8.4 9.0+ 8.3 pg_migrator pg_migrator(1) pg_upgrade 8.4 x pg_migrator pg_upgrade 9.0+ x x pg_upgrade 1. Unable to migrate composites, arrays, and enums. Rapid Upgrades With Pg-Upgrade 19
  20. 20. Conclusion http://momjian.us/presentations Rapid Upgrades With Pg-Upgrade 20

×