The autovacuum and how to tune it to make it suitable for production us cases.

1. Tuning Autovacuum in PostgreSQL
Mohammad Zaid Patel
Database Consultant
Mydbops
Mydbops 16th MyWebinar

2. • Database Consultant for PostgreSQL
• Active Learner
• Technophile
• Intrigued in PostgreSQL and open source software
• Likes Cricket & Music
About Me

3. • Founded in 2016
• Services on top open source databases
• 80+ Member team
• AWS Partner
• PCI & ISO Certified Organisation
About Mydbops

4. Database
Consulting
Services
Managed
Database
Services
Focuses on Top Opensource database MySQL,MariaDB, MongoDB and
PostgreSQL both ON Premises and Cloud
Mydbops Services

5. 500 + Clients In 6 Yrs. of Operations
Our Clients

6. Follow us on LinkedIN !

7. Agenda
• Database Bloat
• What is Vacuum
• What is Autovacuum
• Why Autovacuum
• Tuning Autovacuum
• Best Practices

8. ID NAME
1 TONY
2 STEVE
3 THOR
AVENGERS
>> UPDATE AVENGERS SET NAME='BRUCE' WHERE ID=3 ;
ID NAME
1 TONY
2 STEVE
3 BRUCE
ID NAME
1 TONY
2 STEVE
3 THOR
3 BRUCE
• SELECTS, INSERTS, UPDATES and DELETES are some of the common actions in
any database
• UPDATE = Deletion of Old value + Insertion of New Value
User A
Introduction

9. • Rows marked invisible or as 'dead tuple'
• UNDO log maintained inside the table
• No space reclaimed
• Dead tuples accumulation
Introduction

10. PostgreSQL's MVCC model !

11. ID NAME
1 TONY
2 STEVE
3 THOR
3 BRUCE
ID NAME
1 TONY
2 STEVE
3 BRUCE
User A
User B
>> SELECT NAME FROM AVENGERS WHERE ID=3 ;
ID NAME
1 TONY
2 STEVE
3 THOR
ID NAME
3 THOR
User B
ID NAME
1 TONY
2 STEVE
3 THOR
MVCC model

12. • Data consistency is maintained
• Past image and latest image
• Every row of PostgreSQL table has a version number
• Transaction isolation for each database session
• Locks acquired for querying data don't conflict with locks
acquired for writing
MVCC model

13. Database Bloat

14. Query to check bloat :
postgres=# SELECT pg_size_pretty(pg_relation_size('test')) as
table_size,pgstattuple('test')).dead_tuple_percent;
• Bloating of the Database is caused due to dead tuples
• Increased Dead Tuples = Increased Bloat
Database bloat

15. Why is bloat bad ?
• Slow Sequential scans
• Transaction id wraparound
• Eats up Storage
• Dead tuples > Live tuple = No more updates on the table

16. ID NAME
3 THOR
Xmin Xmax
1 10
Postgres uses 32 bit Transaction ID i.e 2^32 transaction IDs
Transaction ID wraparound

17. Transaction ID wraparound

18. • Limited Transaction IDs
• Once upper limit is reached , ids get wraparound
• Difficult for Postgres to figure out if data is transaction is in past or future
• Postgres shuts down daatabase to protect the data
• Warnings by Postgres
• Very Tedious to bring the database back
Transaction ID wraparound

19. How to remove bloat ?
VACUUM IT !

20. What is Vacuum ?

21. • An important PostgreSQL feature
• Removes the dead tuples
• Reclaims storage
• Various options
Vacuum

22. syntax :
VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ table_and_columns [, ...] ]
Vacuum

23. VACUUM FULL :
• Reclaimed storage is returned back to the OS
• High on resource consumption
• Holds up lock
• Slower than Vacuum
• Requires extra disk space
Vacuum

24. Vacuum
VACUUM FREEZE :
• Marks a table's contents with a very special transaction timestamp
• Freezes committed transaction IDs
• Next update will unfreeze it.

25. VERBOSE :
• Prints a detailed vacuum activity report for each table
Vacuum

26. Vacuum
ANALYZE :
• Updates statistics used by the planner
• Determines the most efficient way to execute a query

27. Is VACUUM perfect solution ?

28. • Manual
• High in Cost
• Employs Locking
• Needs extra disk space
Downsides of Vacuum

29. We've got Autovacuum !

30. What is Autovacuum ?

31. • Released with Postgresql version 8.3
• Background utility
• Automate the execution of VACUUM and ANALYZE
• Not required to be triggered manually
• Does not free up the space
• Reallocates deleted blocks
Autovacuum

32. The autovacuum daemon consists of multiple processes :
• Autovacuum launcher
• Distributes the work across time, attempting to start one worker within each database
every 'autovacuum_nap' seconds
• Maximum of 'autovacuum_max_worker' worker processes are allowed to run at the
same time
• Checks each table within its database and execute VACUUM and/or ANALYZE as
needed
• Functionality of Autovacuum depends on the settings available inside postgresql.conf
file.
Autovacuum

33. Disabling AUTOVACUUM on a Table:
ALTER TABLE table_name SET (autovacuum_enabled = false);
Autovacuum

34. Why Autovacuum ?

35. • Table bloating is prevented
• No locking on Tables
• Not resource-intensive
• Better Storage space utilization
• Improves FSM
Advantages of Autovacuum

36. • Prevents transaction id wraparound
• Faster query execution
• Faster sequential scans
• Better performance of Database
Advantages of Autovacuum

37. TUNING AUTOVACUUM

38. Tuning Autovacuum

39. autovacuum = ON
autovacuum = OFF
track_counts = on
Tuning Autovacuum

40. autovacuum_vacumm_threshold :
Specifies the minimum number of updated or deleted tuples needed to trigger a VACUUM in any
one table
Tuning Autovacuum

41. Tuning Autovacuum
autovacuum_analyze_threshold :
Specifies the minimum number of inserted, updated or deleted tuples needed to trigger
an ANALYZE in any one table

42. autovacuum_vacuum_scale_factor:
Specifies a fraction of the table size to add to autovacumm_vacuum_threshold when
deciding whether to trigger a VACUUM
Tuning Autovacuum

43. Tuning Autovacuum
autovacuum_analyze_scalefactor:
Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding
whether to trigger an ANALYZE

44. autovacuum_naptime:
Minimum time that postgresql wait between each auto vacuum.
Tuning Autovacuum

45. Tuning Autovacuum
autovacuum_max_workers:
Specifies the maximum number of autovacuum processes

46. autovacuum_vacuum_cost_limit:
Specifies the cost delay value that will be used in automatic VACUUM operations after cost
limit is reached.
Tuning Autovacuum

47. Tuning Autovacuum
autovacuum_vacuum_cost_delay:
Specifies the cost limit value that will be used in automatic VACUUM operations

48. Tuning Autovacuum
max_parallel_maintenance_workers:
Sets the maximum number of parallel workers that can be started by a single utility
command

49. Tuning Autovacuum
hot_standby_feedback:
Specifies whether or not a hot standby will send feedback to the primary or upstream
standby about queries currently executing on the standby

50. • autovacuum = on
• autovacuum_vacuum_threshold = 50 , default
• autovacuum_analyze_threshold = 50 , default
• autovacuum_vacuum_scale_factor = 0.20 , default
• autovacuum_analyze_scale_factor = 0.20 , default
• autovacuum_naptime = 60 , default
Parameters Overview

51. • autovacuum_max_workers = 3, default
• autovacuum_vacuum_cost_limit = -1
• autovacuum_vacuum_cost_delay = 2ms
• max_parallel_maintenance_workers = 2
• hot_standby_feedback = on
Parameters Overview

52. Best practises

53. • Detecting Bloat
• Configuring Autovacuum suitable for most of the tables
• Maintenance windows
• Removing bloat for Indexes
• Avoid running manual vacuum/analyze for no reason
• Considering important and critical queries
• Performance after vacuum/analyze should be studied
Best Practises

54. Reach Us : Info@mydbops.com
Thank You