The document discusses pg_osc, a tool for making PostgreSQL schema changes with minimal locking and zero downtime, similar to MySQL's pt_osc. It outlines the step-by-step process for implementing schema changes, including creating audit tables, shadow tables, and handling data changes, while highlighting key features and considerations. The presentation was part of PGConf India on February 29, 2024.

1. PostgreSQL Schema Changes with
pg_osc
PGConf India, 29th Feb 2024
- Aakash M

2. About Us

3. Our Services
Focus on MySQL, MongoDB, PostgreSQL, TiDB, Cassandra
Consulting
Services
Consulting
Services
Managed
Support
Services
24*7
DBA Team
Targeted
Engagement

4. What is pg_osc ?
● Tool for making schema changes
● Minimal lock and zero downtime
● Similar to pt_osc (MySQL) and pg_repack
● Not an extension

5. Problems with Alter
● ALTER - Requires locks, downtime
● CDC - Complex to use

6. Requirement to Use
● PostgreSQL version >= 9.6
● DB User with trigger access or a Superuser

7. Installation
● Using gem command
● Docker Installation

8. How does it work ?
1. Create an Audit table
Audit table

9. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
C1 C2 C3
Primary table
How does it work ?

10. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
C1 C2 C3
C1 C2 C3
Primary table
Shadow table
How does it work ?

11. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
C1 C2 C3 C4
ALTER TABLE SHADOW
TABLE ADD COLUMN C4 ;
Shadow table
How does it work ?

12. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
C1 C2 C3
C1 C2 C3 C4
Shadow table
Primary table
How does it work ?

13. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
C1 C2 C3 C4
Shadow table
CREATING INDEXES
How does it work ?

14. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
C1 C2 C3 C4
Shadow table
Audit table
How does it work ?

15. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
8. Finally Swap table inside a transaction
How does it work ?
C1 C2 C3
C1 C2 C3 C4
Shadow table
Primary table

16. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
8. Finally Swap table inside a transaction
9. Analyze on new table
How does it work ?
C1 C2 C3 C4
ANALYZE ON NEW PRIMARY TABLE

17. 1. Create an Audit table
2. ACCESS EXCLUSIVE lock to add a trigger
3. Create a new shadow table
4. Execute ALTER on shadow table
5. Copy Existing Data
6. Create indexes
7. Replay changes recorded on Audit table
8. Finally Swap table inside a transaction
9. Analyze on new table
10. Drop old table(optional)
How does it work ?
C1 C2 C3
DROPPING THE OLD PRIMARY TABLE

18. Flow Chart

19. Prominent Features
● Minimal Lock throughout the process
● No chance of data loss
● Table rebuild, reduce bloat
● Finetune Replay
● Custom SQL to copy data

20. Few Considerations
● Mandate to have primary key
● No support for Partitioned Table
● Need minimal locks
● Need of more storage
● Index, constraint, Sequence name lose their naming
● Avoid parallel DDLs

21. References
● GitHub : shayonj/pg-osc : https://github.com/shayonj/pg-osc
● Revolutionizing PostgreSQL Schema Changes with pg_osc :
https://www.mydbops.com/blog/postgresql-schema-changes-with-pg_osc/

22. Thank You !
PgConf India 2024