SlideShare a Scribd company logo
1 of 22
The Essential
postgresql.conf
Robert Treat
Beijing Perl Workshop, China, 2008
Introduction



       Robert Treat
       DBA, Postgres Major Contributor
       http://www.xzilla.net/writings.html




       OmniTI
       Internet Scalability Experts
       http://omniti.com/is/hiring
The Grand Scheme



  Understanding the postgresql.conf
  Essential postgresql.conf parameters
The Grand Scheme



  Understanding the postgresql.conf
  Essential postgresql.conf parameters
Understanding the postgresql.conf

 Types of settings
 When settings take effect
 postgresql.conf gotchas
 Viewing your current settings
Types of Settings
    ●   Boolean : true, false, on, off
    ●   Integer : whole numbers (2112)
    ●   Float : decimal values (21.12)
    ●   Memory/Disk Units : 128MB, 2112GB
         ●   Avoid using integers to size memory params
    ●   Time : d,m,s (30s)
    ●   Strings : single quotes ('pg_log')
    ●   Enum : single quotes ('WARNING' or 'ERROR')
             ●   Limited set of acceptable strings
    ●   Lists : comma separated ('”$user”,public,tsearch2')
When Settings Take Effect

       ●   Internal : set at compile time
       ●   Postmaster : requires database restart
       ●   Backend : must be set before session start
       ●   Sighup : change by reloading conf file
           ●
               pg_reload_conf(), pg_ctl reload, kill -HUP
       ●   Superuser : set at runtime by superuser
       ●   User : set at runtime, effects only single session


 See pg_settings.context to determine setting type
When Settings Take Effect

       ●   Internal : set at compile time
       ●   Postmaster : requires database restart
       ●   Backend : must be set before session start
       ●   Sighup : change by reloading conf file
           ●
               pg_reload_conf(), pg_ctl reload, kill -HUP
       ●   Superuser : set at runtime by superuser
       ●   User : set at runtime, effects only single session


 See pg_settings.context to determine setting type
postgresql.conf Gotchas!

    ●   $PGDATA/postgresql.conf
        ●   Watch out for symbolic links
        ●   Some distros put it other places (/etc)
    ●   Lines with #
        ●   Line is a comment, has no effect
        ●   On fresh install, uses default
        ●   On <8.3, comment a line does not restore the default
    ●   Settings listed multiple times
        ●   Last one wins
View Your Current Settings
    ●   Look in postgresql.conf
        ●   Works if you follow best practice
        ●   Not definitive!
    ●   Show all, show <setting>
        ●   Shows current value
        ●   Watch out for session specific changes
    ●   Select * from pg_settings
        ●   Shows current value
        ●
            “source” column shows if session specific
Essential postgresql.conf parameters

 Getting connected
 Logging
 Sizing Memory
 WAL Management
 Checkpoint Management
 Vacuuming
 Leftovers
Getting Connected
   ●   listen_addresses
       ●   Requires restart!
       ●   Default is localhost only
       ●   Use * to enable tcp/ip
   ●   max_connections
       ●   Requires restart!
       ●   100 default (apache children default is higher)
       ●   Affects other settings (work_mem)
       ●   For 1000+ connections, use connection pool
Logging
   ●   log_destination, log_directory, log_filename
       ●   Find out where you are logging
       ●   Might be good to log to different disk than data
   ●   log_min_error_statement
       ●   Make sure set to at least 'ERROR'
   ●   log_line_prefix
       ●   Generic recommendation: '%t:%r:%u@%d:[%p]: '
       ●   Timestamp, connecting host, username, database, pid
Sizing Memory
   ●   shared_buffers
       ●   Requires Restart!
       ●   Most important setting for good performance
       ●   Allocation of memory for Postgres caching
       ●   ~ 20% of Total RAM (up to 2GB)
            ●   32GB of RAM, 1920MB probably ok
            ●   Windows users probably want less
            ●   May require adjusting kernel params (shmmax)
                   ●   http://www.postgresql.org/docs/current/interactive/kernel-resources.html
Sizing Memory
   ●   effective_cache_size
       ●   Most important setting for good performance
       ●   Based on memory available for filesystem cache
       ●   SET = TRAM – (DRAM – PGRAM)
            ●   TRAM = Total RAM
            ●   DRAM = RAM needed by OS and other applications
            ●   PGRAM = RAM needed for Postgres (shared_buffers, etc...)
       ●   Guide for available memory, not an allocation
Sizing Memory
   ●   work_mem
           ●   Used by queries for sorting
           ●   Higher values for more complex queries
           ●   Limit is Per sort
                ●   5 sorts in 1 query = 5*work_mem
   ●   maintenance_work_mem
       ● Used by vacuum, indexing, and similar operations
       ● Semi-Allocation of memory when needed

       ● Values up to 128MB have been found useful
WAL Management
   ●   wal_buffers
       ●   Requires restart!
       ●   needed for write-heavy databases
       ●   1024kB is good starting point
   ●   wal_sync_method
       ●   WAL is sync'd after every transaction
       ●   fsync is ok default, but tuning can be helpful sometimes
       ●   http://www.westnet.com/~gsmith/content/postgresql/TuningPGWAL.htm
   ●   synchronous_commit
       ●   Can increase tps by large margin
       ●   Allows data loss without corruption
            ●   BAD:  fsync=off
            ●   GOOD: synchronous_commit=off
       ●   Set per database, user, or transaction
Checkpoint Management
   ●   checkpoint_segments
       ●   Controls amount of data required per checkpoint
       ●   Recommend default of 10
       ●   Set higher for heavy write activity
       ●   30-300 is not uncommon
       ●   Increases space needed in pg_xlog
       ●   Increases recovery time after crash
   ●   checkpoint_completion_target
       ●   With checkpoint_segments >= 10, set this to .9
       ●   http://www.westnet.com/~gsmith/content/postgresql/chkp-bgw-83.htm
Vacuum Management
   ●   autovacuum
       ●   Just do it! (Default on 8.3+)
   ●   default_statistics_target
       ●   Defines amount of data collected on tables for planner
       ●   Important for good query planning
       ●   100 is a better default (might need further tuning)
Vacuum Management
   ●   max_fsm_relations, max_fsm_pages
       ●   Both Require Restart!
       ●   Track dead tuples in database cluster
           ●   fsm_relations – number of tables in all databases
           ●   fsm_pages – number of pages with dead tuples in all databases
       ●   “vacuum verbose” in postgres database
                 INFO: free space map contains 5293 pages in 214 relations
                 DETAIL: A total of 8528 page slots are in use (including overhead).
                 8528 page slots are required to track all free space.
                 Current limits are: 204800 page slots, 1000 relations, using 1265 kB

       ●   check_postgres script has a check for both
           ●   http://bucardo.org/check_postgres/
           ●   Requires pg_freespacemap from contrib
       ●   These go away in Postgres 8.4 (yay!)
Leftovers
    ●   random_page_cost
        ●   For server grade hardware, most set to 3
    ●   max_prepared_transactions
        ●   Can be set to zero if not using prepared transactions
    ●   constraint_exclusion
        ●   Needed for table partitioning
        ●   Good for setting at database or query level
The End
   ●   Additional Resources
       ●   http://www.postgresql.org/docs/8.3/interactive/runtime-config.html
       ●   http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server
   ●   Slides
       ●   http://www.xzilla.net/writings.html
   ●   Xìe xìe

More Related Content

What's hot

plProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancerplProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancer
elliando dias
 
Advanced Postgres Monitoring
Advanced Postgres MonitoringAdvanced Postgres Monitoring
Advanced Postgres Monitoring
Denish Patel
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
PostgreSQL-Consulting
 
006 performance tuningandclusteradmin
006 performance tuningandclusteradmin006 performance tuningandclusteradmin
006 performance tuningandclusteradmin
Scott Miao
 

What's hot (19)

plProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancerplProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancer
 
Advanced Postgres Monitoring
Advanced Postgres MonitoringAdvanced Postgres Monitoring
Advanced Postgres Monitoring
 
Troubleshooting PostgreSQL Streaming Replication
Troubleshooting PostgreSQL Streaming ReplicationTroubleshooting PostgreSQL Streaming Replication
Troubleshooting PostgreSQL Streaming Replication
 
GUC Tutorial Package (9.0)
GUC Tutorial Package (9.0)GUC Tutorial Package (9.0)
GUC Tutorial Package (9.0)
 
Streaming replication in PostgreSQL
Streaming replication in PostgreSQLStreaming replication in PostgreSQL
Streaming replication in PostgreSQL
 
PostgreSQL Replication Tutorial
PostgreSQL Replication TutorialPostgreSQL Replication Tutorial
PostgreSQL Replication Tutorial
 
Managing PostgreSQL with PgCenter
Managing PostgreSQL with PgCenterManaging PostgreSQL with PgCenter
Managing PostgreSQL with PgCenter
 
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
How does PostgreSQL work with disks: a DBA's checklist in detail. PGConf.US 2015
 
Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4Out of the box replication in postgres 9.4
Out of the box replication in postgres 9.4
 
Connection Pooling in PostgreSQL using pgbouncer
Connection Pooling in PostgreSQL using pgbouncer Connection Pooling in PostgreSQL using pgbouncer
Connection Pooling in PostgreSQL using pgbouncer
 
Shootout at the PAAS Corral
Shootout at the PAAS CorralShootout at the PAAS Corral
Shootout at the PAAS Corral
 
Troubleshooting PostgreSQL with pgCenter
Troubleshooting PostgreSQL with pgCenterTroubleshooting PostgreSQL with pgCenter
Troubleshooting PostgreSQL with pgCenter
 
Pgcenter overview
Pgcenter overviewPgcenter overview
Pgcenter overview
 
GitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons LearnedGitLab PostgresMortem: Lessons Learned
GitLab PostgresMortem: Lessons Learned
 
Tuning Linux for Databases.
Tuning Linux for Databases.Tuning Linux for Databases.
Tuning Linux for Databases.
 
Как PostgreSQL работает с диском
Как PostgreSQL работает с дискомКак PostgreSQL работает с диском
Как PostgreSQL работает с диском
 
Streaming Replication Made Easy in v9.3
Streaming Replication Made Easy in v9.3Streaming Replication Made Easy in v9.3
Streaming Replication Made Easy in v9.3
 
006 performance tuningandclusteradmin
006 performance tuningandclusteradmin006 performance tuningandclusteradmin
006 performance tuningandclusteradmin
 
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
PostgreSQL Troubleshoot On-line, (RITfest 2015 meetup at Moscow, Russia).
 

Similar to The Essential postgresql.conf

Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)
Denish Patel
 

Similar to The Essential postgresql.conf (20)

PostgreSQL Prologue
PostgreSQL ProloguePostgreSQL Prologue
PostgreSQL Prologue
 
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
PuppetConf 2016: An Introduction to Measuring and Tuning PE Performance – Cha...
 
High Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniHigh Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando Patroni
 
PostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter TuningPostgreSQL Hangout Parameter Tuning
PostgreSQL Hangout Parameter Tuning
 
OpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQLOpenGurukul : Database : PostgreSQL
OpenGurukul : Database : PostgreSQL
 
MySQL Tuning
MySQL TuningMySQL Tuning
MySQL Tuning
 
♨️CPU limitation per Oracle database instance
♨️CPU limitation per Oracle database instance♨️CPU limitation per Oracle database instance
♨️CPU limitation per Oracle database instance
 
The Accidental DBA
The Accidental DBAThe Accidental DBA
The Accidental DBA
 
Oracle to Postgres Migration - part 2
Oracle to Postgres Migration - part 2Oracle to Postgres Migration - part 2
Oracle to Postgres Migration - part 2
 
Filesystem Performance from a Database Perspective
Filesystem Performance from a Database PerspectiveFilesystem Performance from a Database Perspective
Filesystem Performance from a Database Perspective
 
How to Replicate PostgreSQL Database
How to Replicate PostgreSQL DatabaseHow to Replicate PostgreSQL Database
How to Replicate PostgreSQL Database
 
guc_tutorial_10.pdf
guc_tutorial_10.pdfguc_tutorial_10.pdf
guc_tutorial_10.pdf
 
Out of the box replication in postgres 9.4(pg confus)
Out of the box replication in postgres 9.4(pg confus)Out of the box replication in postgres 9.4(pg confus)
Out of the box replication in postgres 9.4(pg confus)
 
Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)Out of the Box Replication in Postgres 9.4(PgConfUS)
Out of the Box Replication in Postgres 9.4(PgConfUS)
 
AutoDOPandRest
AutoDOPandRestAutoDOPandRest
AutoDOPandRest
 
PostgreSQL Replication in 10 Minutes - SCALE
PostgreSQL Replication in 10  Minutes - SCALEPostgreSQL Replication in 10  Minutes - SCALE
PostgreSQL Replication in 10 Minutes - SCALE
 
10 things i wish i'd known before using spark in production
10 things i wish i'd known before using spark in production10 things i wish i'd known before using spark in production
10 things i wish i'd known before using spark in production
 
Clug 2012 March web server optimisation
Clug 2012 March   web server optimisationClug 2012 March   web server optimisation
Clug 2012 March web server optimisation
 
PostgreSQL Configuration for Humans / Alvaro Hernandez (OnGres)
PostgreSQL Configuration for Humans / Alvaro Hernandez (OnGres)PostgreSQL Configuration for Humans / Alvaro Hernandez (OnGres)
PostgreSQL Configuration for Humans / Alvaro Hernandez (OnGres)
 
Postgresql Database Administration Basic - Day1
Postgresql  Database Administration Basic  - Day1Postgresql  Database Administration Basic  - Day1
Postgresql Database Administration Basic - Day1
 

More from Robert Treat

Less Alarming Alerts!
Less Alarming Alerts!Less Alarming Alerts!
Less Alarming Alerts!
Robert Treat
 
Managing Databases In A DevOps Environment
Managing Databases In A DevOps EnvironmentManaging Databases In A DevOps Environment
Managing Databases In A DevOps Environment
Robert Treat
 
Scaling with Postgres (Highload++ 2010)
Scaling with Postgres (Highload++ 2010)Scaling with Postgres (Highload++ 2010)
Scaling with Postgres (Highload++ 2010)
Robert Treat
 
Intro to Postgres 9 Tutorial
Intro to Postgres 9 TutorialIntro to Postgres 9 Tutorial
Intro to Postgres 9 Tutorial
Robert Treat
 

More from Robert Treat (20)

Advanced Int->Bigint Conversions
Advanced Int->Bigint ConversionsAdvanced Int->Bigint Conversions
Advanced Int->Bigint Conversions
 
Explaining Explain
Explaining ExplainExplaining Explain
Explaining Explain
 
the-lost-art-of-plpgsql
the-lost-art-of-plpgsqlthe-lost-art-of-plpgsql
the-lost-art-of-plpgsql
 
Managing Chaos In Production: Testing vs Monitoring
Managing Chaos In Production: Testing vs MonitoringManaging Chaos In Production: Testing vs Monitoring
Managing Chaos In Production: Testing vs Monitoring
 
Managing Databases In A DevOps Environment 2016
Managing Databases In A DevOps Environment 2016Managing Databases In A DevOps Environment 2016
Managing Databases In A DevOps Environment 2016
 
Less Alarming Alerts - SRECon 2016
Less Alarming Alerts - SRECon 2016 Less Alarming Alerts - SRECon 2016
Less Alarming Alerts - SRECon 2016
 
What Ops Can Learn From Design
What Ops Can Learn From DesignWhat Ops Can Learn From Design
What Ops Can Learn From Design
 
Postgres 9.4 First Look
Postgres 9.4 First LookPostgres 9.4 First Look
Postgres 9.4 First Look
 
Less Alarming Alerts!
Less Alarming Alerts!Less Alarming Alerts!
Less Alarming Alerts!
 
Past, Present, and Pachyderm - All Things Open - 2013
Past, Present, and Pachyderm - All Things Open - 2013Past, Present, and Pachyderm - All Things Open - 2013
Past, Present, and Pachyderm - All Things Open - 2013
 
Big Bad "Upgraded" Postgres
Big Bad "Upgraded" PostgresBig Bad "Upgraded" Postgres
Big Bad "Upgraded" Postgres
 
Managing Databases In A DevOps Environment
Managing Databases In A DevOps EnvironmentManaging Databases In A DevOps Environment
Managing Databases In A DevOps Environment
 
The Essential PostgreSQL.conf
The Essential PostgreSQL.confThe Essential PostgreSQL.conf
The Essential PostgreSQL.conf
 
Pro Postgres 9
Pro Postgres 9Pro Postgres 9
Pro Postgres 9
 
Advanced WAL File Management With OmniPITR
Advanced WAL File Management With OmniPITRAdvanced WAL File Management With OmniPITR
Advanced WAL File Management With OmniPITR
 
Scaling with Postgres (Highload++ 2010)
Scaling with Postgres (Highload++ 2010)Scaling with Postgres (Highload++ 2010)
Scaling with Postgres (Highload++ 2010)
 
Intro to Postgres 9 Tutorial
Intro to Postgres 9 TutorialIntro to Postgres 9 Tutorial
Intro to Postgres 9 Tutorial
 
Check Please!
Check Please!Check Please!
Check Please!
 
Database Scalability Patterns
Database Scalability PatternsDatabase Scalability Patterns
Database Scalability Patterns
 
A Guide To PostgreSQL 9.0
A Guide To PostgreSQL 9.0A Guide To PostgreSQL 9.0
A Guide To PostgreSQL 9.0
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 

Recently uploaded (20)

Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
The UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, OcadoThe UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, Ocado
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Buy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptxBuy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptx
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 

The Essential postgresql.conf

  • 2. Introduction Robert Treat DBA, Postgres Major Contributor http://www.xzilla.net/writings.html OmniTI Internet Scalability Experts http://omniti.com/is/hiring
  • 3. The Grand Scheme Understanding the postgresql.conf Essential postgresql.conf parameters
  • 4. The Grand Scheme Understanding the postgresql.conf Essential postgresql.conf parameters
  • 5. Understanding the postgresql.conf Types of settings When settings take effect postgresql.conf gotchas Viewing your current settings
  • 6. Types of Settings ● Boolean : true, false, on, off ● Integer : whole numbers (2112) ● Float : decimal values (21.12) ● Memory/Disk Units : 128MB, 2112GB ● Avoid using integers to size memory params ● Time : d,m,s (30s) ● Strings : single quotes ('pg_log') ● Enum : single quotes ('WARNING' or 'ERROR') ● Limited set of acceptable strings ● Lists : comma separated ('”$user”,public,tsearch2')
  • 7. When Settings Take Effect ● Internal : set at compile time ● Postmaster : requires database restart ● Backend : must be set before session start ● Sighup : change by reloading conf file ● pg_reload_conf(), pg_ctl reload, kill -HUP ● Superuser : set at runtime by superuser ● User : set at runtime, effects only single session See pg_settings.context to determine setting type
  • 8. When Settings Take Effect ● Internal : set at compile time ● Postmaster : requires database restart ● Backend : must be set before session start ● Sighup : change by reloading conf file ● pg_reload_conf(), pg_ctl reload, kill -HUP ● Superuser : set at runtime by superuser ● User : set at runtime, effects only single session See pg_settings.context to determine setting type
  • 9. postgresql.conf Gotchas! ● $PGDATA/postgresql.conf ● Watch out for symbolic links ● Some distros put it other places (/etc) ● Lines with # ● Line is a comment, has no effect ● On fresh install, uses default ● On <8.3, comment a line does not restore the default ● Settings listed multiple times ● Last one wins
  • 10. View Your Current Settings ● Look in postgresql.conf ● Works if you follow best practice ● Not definitive! ● Show all, show <setting> ● Shows current value ● Watch out for session specific changes ● Select * from pg_settings ● Shows current value ● “source” column shows if session specific
  • 11. Essential postgresql.conf parameters Getting connected Logging Sizing Memory WAL Management Checkpoint Management Vacuuming Leftovers
  • 12. Getting Connected ● listen_addresses ● Requires restart! ● Default is localhost only ● Use * to enable tcp/ip ● max_connections ● Requires restart! ● 100 default (apache children default is higher) ● Affects other settings (work_mem) ● For 1000+ connections, use connection pool
  • 13. Logging ● log_destination, log_directory, log_filename ● Find out where you are logging ● Might be good to log to different disk than data ● log_min_error_statement ● Make sure set to at least 'ERROR' ● log_line_prefix ● Generic recommendation: '%t:%r:%u@%d:[%p]: ' ● Timestamp, connecting host, username, database, pid
  • 14. Sizing Memory ● shared_buffers ● Requires Restart! ● Most important setting for good performance ● Allocation of memory for Postgres caching ● ~ 20% of Total RAM (up to 2GB) ● 32GB of RAM, 1920MB probably ok ● Windows users probably want less ● May require adjusting kernel params (shmmax) ● http://www.postgresql.org/docs/current/interactive/kernel-resources.html
  • 15. Sizing Memory ● effective_cache_size ● Most important setting for good performance ● Based on memory available for filesystem cache ● SET = TRAM – (DRAM – PGRAM) ● TRAM = Total RAM ● DRAM = RAM needed by OS and other applications ● PGRAM = RAM needed for Postgres (shared_buffers, etc...) ● Guide for available memory, not an allocation
  • 16. Sizing Memory ● work_mem ● Used by queries for sorting ● Higher values for more complex queries ● Limit is Per sort ● 5 sorts in 1 query = 5*work_mem ● maintenance_work_mem ● Used by vacuum, indexing, and similar operations ● Semi-Allocation of memory when needed ● Values up to 128MB have been found useful
  • 17. WAL Management ● wal_buffers ● Requires restart! ● needed for write-heavy databases ● 1024kB is good starting point ● wal_sync_method ● WAL is sync'd after every transaction ● fsync is ok default, but tuning can be helpful sometimes ● http://www.westnet.com/~gsmith/content/postgresql/TuningPGWAL.htm ● synchronous_commit ● Can increase tps by large margin ● Allows data loss without corruption ● BAD: fsync=off ● GOOD: synchronous_commit=off ● Set per database, user, or transaction
  • 18. Checkpoint Management ● checkpoint_segments ● Controls amount of data required per checkpoint ● Recommend default of 10 ● Set higher for heavy write activity ● 30-300 is not uncommon ● Increases space needed in pg_xlog ● Increases recovery time after crash ● checkpoint_completion_target ● With checkpoint_segments >= 10, set this to .9 ● http://www.westnet.com/~gsmith/content/postgresql/chkp-bgw-83.htm
  • 19. Vacuum Management ● autovacuum ● Just do it! (Default on 8.3+) ● default_statistics_target ● Defines amount of data collected on tables for planner ● Important for good query planning ● 100 is a better default (might need further tuning)
  • 20. Vacuum Management ● max_fsm_relations, max_fsm_pages ● Both Require Restart! ● Track dead tuples in database cluster ● fsm_relations – number of tables in all databases ● fsm_pages – number of pages with dead tuples in all databases ● “vacuum verbose” in postgres database INFO: free space map contains 5293 pages in 214 relations DETAIL: A total of 8528 page slots are in use (including overhead). 8528 page slots are required to track all free space. Current limits are: 204800 page slots, 1000 relations, using 1265 kB ● check_postgres script has a check for both ● http://bucardo.org/check_postgres/ ● Requires pg_freespacemap from contrib ● These go away in Postgres 8.4 (yay!)
  • 21. Leftovers ● random_page_cost ● For server grade hardware, most set to 3 ● max_prepared_transactions ● Can be set to zero if not using prepared transactions ● constraint_exclusion ● Needed for table partitioning ● Good for setting at database or query level
  • 22. The End ● Additional Resources ● http://www.postgresql.org/docs/8.3/interactive/runtime-config.html ● http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server ● Slides ● http://www.xzilla.net/writings.html ● Xìe xìe