The Essential PostgreSQL.conf

10,874 views
10,678 views

Published on

This talk cover the most important postgresql.conf variables that users should tune and/or study up on. It is geared toward Postgres 9.0.x servers.

Published in: Technology
1 Comment
23 Likes
Statistics
Notes
No Downloads
Views
Total views
10,874
On SlideShare
0
From Embeds
0
Number of Embeds
32
Actions
Shares
0
Downloads
0
Comments
1
Likes
23
Embeds 0
No embeds

No notes for slide

The Essential PostgreSQL.conf

  1. 1. The Essential PostgreSQL.conf 9 Robert Treat, JDCon East 2011Thursday, March 31, 2011
  2. 2. Introduction • Robert
Treat • OmniTI • Database
Management
and
Consulting • Large
and/or
High
Traffic
Architectures • We’re
Hiring! • Postgres • Web,

Advocacy,
Conferences,
PhpPgAdmin • Major
Contributor • xzilla.net • @robtreat2Thursday, March 31, 2011
  3. 3. Introduction • Robert
Treat • OmniTI • Database
Management
and
Consulting • Large
and/or
High
Traffic
Architectures • We’re
Hiring! • Postgres • Web,

Advocacy,
Conferences,
PhpPgAdmin • Major
Contributor • xzilla.net • @robtreat2Thursday, March 31, 2011
  4. 4. The Grand Scheme • Understanding the postgresql.conf • Essential postgresql.conf parametersThursday, March 31, 2011
  5. 5. The Grand Scheme • Understanding the postgresql.conf • Essential postgresql.conf parametersThursday, March 31, 2011
  6. 6. Understanding the postgresql.conf • Types of Settings • When Settings Take Effect • PostgreSQL.conf “Gotchas” • Viewing Your Current SettingsThursday, March 31, 2011
  7. 7. Types of SettingsThursday, March 31, 2011
  8. 8. Types of Settings • Boolean: true, false, on, offThursday, March 31, 2011
  9. 9. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112)Thursday, March 31, 2011
  10. 10. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12)Thursday, March 31, 2011
  11. 11. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12) • Memory/Disk Units: 128MB, 2112GBThursday, March 31, 2011
  12. 12. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12) • Memory/Disk Units: 128MB, 2112GB • Avoid using integers to size memory parametersThursday, March 31, 2011
  13. 13. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12) • Memory/Disk Units: 128MB, 2112GB • Avoid using integers to size memory parameters • Time: d,m,s (30s)Thursday, March 31, 2011
  14. 14. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12) • Memory/Disk Units: 128MB, 2112GB • Avoid using integers to size memory parameters • Time: d,m,s (30s) • Strings: single quotes (‘pg_log’)Thursday, March 31, 2011
  15. 15. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12) • Memory/Disk Units: 128MB, 2112GB • Avoid using integers to size memory parameters • Time: d,m,s (30s) • Strings: single quotes (‘pg_log’) • Enum: single quotes (‘WARNING’ or ‘ERROR’)Thursday, March 31, 2011
  16. 16. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12) • Memory/Disk Units: 128MB, 2112GB • Avoid using integers to size memory parameters • Time: d,m,s (30s) • Strings: single quotes (‘pg_log’) • Enum: single quotes (‘WARNING’ or ‘ERROR’) • Limited set of acceptable stringsThursday, March 31, 2011
  17. 17. Types of Settings • Boolean: true, false, on, off • Integer: whole numbers (2112) • Float: decimal values (21.12) • Memory/Disk Units: 128MB, 2112GB • Avoid using integers to size memory parameters • Time: d,m,s (30s) • Strings: single quotes (‘pg_log’) • Enum: single quotes (‘WARNING’ or ‘ERROR’) • Limited set of acceptable strings • Lists: comma seperated (‘“$user”,public,tsearch2’)Thursday, March 31, 2011
  18. 18. When Settings Take Effect • Internal: set at compile time • Postmaster: requires database restart • Backend: must be set before session start • Sighup: change by reloading conf file • pg_reload_conf(), pg_ctl reload, kill -HUP • Superuser: set at runtime by superuser • User: set at runtime, affects only a single sessionSee
pg_settings.context
to
determine
setting
typeThursday, March 31, 2011
  19. 19. When Settings Take Effect • Internal: set at compile time • Postmaster: requires database restart • Backend: must be set before session start • Sighup: change by reloading conf file • pg_reload_conf(), pg_ctl reload, kill -HUP • Superuser: set at runtime by superuser • User: set at runtime, affects only a single sessionSee
pg_settings.context
to
determine
setting
typeThursday, March 31, 2011
  20. 20. postgresql.conf Gotchas! • Where is my postgresql.conf ? • $PGDATA/postgresql.conf • watch out for symbolic links • some distros put it in other places (/etc) • SHOW data_directory; • SHOW config_file;Thursday, March 31, 2011
  21. 21. postgresql.conf Gotchas! • Lines with # • This line is a comment, and has no effect • On a fresh install, uses defaults • Commenting a line is the same as deleting a line, it will restore the “default” for that instanceThursday, March 31, 2011
  22. 22. Viewing Your Current Settings • Look in postgresql.conf • Works if you follow best practices • Not definitive!Thursday, March 31, 2011
  23. 23. Viewing Your Current Settings • Look in postgresql.conf • Works if you follow best practices • Not definitive! • SHOW ALL, SHOW <setting>, current_setting(setting) • Shows the current value • Watch out for session specific changesThursday, March 31, 2011
  24. 24. Viewing Your Current Settings • Look in postgresql.conf • Works if you follow best practices • Not definitive! • SHOW ALL, SHOW <setting>, current_setting(setting) • Shows the current value • Watch out for session specific changes • SELECT * FROM pg_settings • Shows the current value • “source” columnThursday, March 31, 2011
  25. 25. Viewing Your Current Settings pagila=# select current_setting(name), * from pg_settings where name = shared_buffers; -[ RECORD 1 ]------+---------------------------------------------- current_setting | 1600kB name | shared_buffers setting | 200 unit | 8kB category | Resource Usage / Memory short_desc | Sets the number of shared memory buffers used by the server. extra_desc | context | postmaster vartype | integer source | configuration file min_val | 16 max_val | 1073741823 enumvals | boot_val | 1024 reset_val | 200 sourcefile | /opt/local/var/db/postgresql91/data/postgresql.conf sourceline | 109Thursday, March 31, 2011
  26. 26. The Essential Parameters • Logging • Getting Connected • Sizing Memory • Checkpoint Management • Vacuum Managment • WAL managementThursday, March 31, 2011
  27. 27. Logging • log_destination, log_directory, log_filename • find out where you are logging • might be good to log to a different disk than your dataThursday, March 31, 2011
  28. 28. Logging • log_destination, log_directory, log_filename • find out where you are logging • might be good to log to a different disk than your data • log_line_prefix • generic recommendation: ‘%t: %r: %u@%d: [%p]: ‘ • timestamp, connecting host, username, database, pidThursday, March 31, 2011
  29. 29. Get Connected • listen_addresses • requires restart! • default is localhost only • use * to enable tcp/ipThursday, March 31, 2011
  30. 30. Get Connected • listen_addresses • requires restart! • default is localhost only • use * to enable tcp/ip • max_connections • requires restart! • 100 default (apache children default is higher) • affects other settings (work_mem) • for 1000+ connections, use connection poolerThursday, March 31, 2011
  31. 31. Sizing Memory • shared_buffers • requires restart! • most important setting for good performance • allocation of memory for Postgres caching • ~20% of Total Available RAM (up to 2GB) • 32GB RAM, 1920 MB is probably ok • 256GB RAM, 50 GB is probably not optimal • Windows users probably want less • may require adjusting kernel params (shmmax) • http://www.postgresql.org/docs/current/interactive/kernel-resources.htmlThursday, March 31, 2011
  32. 32. Sizing Memory • effective_cache_size • most important setting for good performance • based on memory available for filesystem cache • SET = TRAM - (DRAM - PGRAM) • TRAM = Total RAM • DRAM = RAM needed by OS and other apps • PGRAM = RAM needed for Postgres (shared_buffers, etc...) • Guide for available memory, not an allocationThursday, March 31, 2011
  33. 33. Sizing Memory • work_mem • used by queries for sorting • higher values for more complex queries • limit is per sort • 5 sorts in 1 query = 5*work_mem • maintenance_work_mem • used by vacuum, indexing, and similar operations • semi-allocation of memory when needed • don’t be scared to make this big (1GB)Thursday, March 31, 2011
  34. 34. Checkpoint Management • checkpoint_segments • controls the amount of data required per checkpoint • default 3, recommended minimum of 10 • set higher for heavy write activity • 30 - 300 is not uncommon • increases space needed in pg_xlog • increases recovery time after crash • checkpoint_completion_target • with checkpoint_segments >= 10, set this to .9 • http://www.westnet.com/~gsmith/content/postgresql/chkp-bgw-83.htmThursday, March 31, 2011
  35. 35. Vacuum Management • autovacuum • max_fsm_pages • max_fsm_relations • default_statistics_target • autovacuum_freeze_max_age • set to 1,000,000,000 to reduce vacuums for non-activity • requires restartThursday, March 31, 2011
  36. 36. WAL Management / Replication • WAL for Performance • wal_buffers • requires restart! • needed for write heavy loads • 16MB, “set it and forget it!” • synchronous_commit • can increase tps by a large margin • allows data loss without corruption • BAD: fsync = off • GOOD: synchronous_commit = off • can be set per database, user, or transactionThursday, March 31, 2011
  37. 37. WAL Management / Replication • WAL for Replication • wal_level • requires restart! • “hot_standby” needed for... hot standbys • max_wal_senders • requires restart! • controls the number of connections for streaming replicationThursday, March 31, 2011
  38. 38. The End Thanks! Additional Resources http://www.postgresql.org/docs/current/interactive/runtime-config.html http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server Slides http://www.xzilla.net/writings.htmlThursday, March 31, 2011
  39. 39. No Really, The EndThursday, March 31, 2011

×