This document provides information about database bloat and performance tuning. It introduces Denish Patel, a database architect with expertise in heterogeneous databases including PostgreSQL, Oracle and MySQL. The document discusses what causes database bloat, issues it can create, and tools for identifying, measuring and removing bloat. These include vacuum, vacuum full, cluster, pg_bloat_report, check_postgres_bloat, compact_table and pg_reorg. Monitoring and prevention techniques are also covered.
4. Who am I ?
• Denish Patel
• Database Architect
• 4+ Years with OmniTi
• Expertise
• Heterogeneous Databases
• PostgreSQL, Oracle, MySQL
• Very Large databases
• Performance tuning
• Application level
• SQL tuning
• Database and OS parameters
• Objects tuning 2
Friday, March 25, 2011
5. Who am I ?
• Denish Patel
• Database Architect
• 4+ Years with OmniTi
• Expertise
• Heterogeneous Databases
• PostgreSQL, Oracle, MySQL
• Very Large databases
• Performance tuning
• Application level OmniTi is hiring!
• SQL tuning
• Database and OS parameters
• Objects tuning 2
Friday, March 25, 2011
36. vacuum
• Auto vacuum
• VACUUM [verbose|analyze] table
• Reclaims storage occupied by dead
tuples
• Reclaimed space available to re-use
• Normally free space doesn’t return to
OS
• No exclusive table lock
15
Friday, March 25, 2011
38. vacuum full
• VACUUM FULL table
• Compact table based on dead rows
• Starting PostgreSQL 9.0 , it’s rewrite
entire table and indexes (like CLUSTER)
• Reclaimed space available to re-use
• Space returned to OS
• Table Exclusive lock
• Expensive operation
16
Friday, March 25, 2011
42. cluster
•CLUSTER [verbose] table_name
[USING index_name]
•Create reorganized copy of table
based on index
•Rebuild indexes too
•Requires Exclusive Lock on table
•Expensive Operation
18
Friday, March 25, 2011
44. cluster vs vacuum full
It’s just rewrite of table without
Order based on Index
any specific order
Table needs index Doesn’t need index
Bloat indexes (Pre PostgreSQL
Doesn’t bloat index
9.0)
Overall Operation Slower (Pre
Overall operation Faster
PostgreSQL 9.0)
Recommended Not recommended
19
Friday, March 25, 2011
48. compact_table
•Developed by OmniTi
•Reorganize rows to empty pages at the
end based on ctid
•ctid - (page number, tuple number)
•When we update a row, the row's ctid
changes, because the update creates a
new version of the row and leaves the
old version behind
21
Friday, March 25, 2011
53. compact_table
postgres=# truncate table bar;
TRUNCATE TABLE
postgres=# insert into bar select generate_series(1,1000);
INSERT 0 1000
postgres=# delete from bar where a %2 = 0;
DELETE 500
postgres=# select max(ctid) from bar;
max
--------
(4,95)
(1 row)
postgres=# vacuum verbose bar;
INFO: "bar": found 0 removable, 500
nonremovable row versions in 5 out of 5 pages
VACUUM
23
Friday, March 25, 2011
55. compact_table
postgres=# begin;
BEGIN
postgres=# update bar set a=a where ctid>='(3,0)';
UPDATE 161
postgres=# update bar set a=a where ctid>='(3,0)';
UPDATE 161
postgres=# update bar set a=a where ctid>='(3,0)';
UPDATE 48
postgres=# update bar set a=a where ctid>='(3,0)';
UPDATE 48
postgres=# update bar set a=a where ctid>='(3,0)';
UPDATE 34
postgres=# update bar set a=a where ctid>='(3,0)';
UPDATE 0
postgres=# commit;
COMMIT
postgres=# vacuum verbose bar;
.
.
INFO: "bar": truncated 5 to 3 pages
.
VACUUM
24
Friday, March 25, 2011
57. compact_table
•Pretty safe update process
•No additional changes required
•Slow process
•Generate Index bloat
•Under development
•Not tested in production
•Future consideration?
25
Friday, March 25, 2011
61. pg_reorg
• Developed by NTT OSS Center
• Reorganize table with very short lock
• Needs Primary Key
• Not Null Unique Key works!
• CLUSTER online
• ORDER BY Key
• VACUUM FULL Online
• Rewrite without ORDER BY
• Build and Install like other Contrib Module
27
Friday, March 25, 2011
105. open discussion
•When PostgreSQL will have built in
features to rebuild table and index
online ?
•Is it worth to change MVCC handling
to avoid bloating?
45
Friday, March 25, 2011