Your SlideShare is downloading. ×
Pro Postgres 9                Robert Treat, JDCon East 2011Thursday, March 31, 2011
Who Am I? (Why Listen To Me)               PostgreSQL User Since 6.5.x               DBA of High Traffic / Large PostgreSQL...
Outline              What you need to know              about the project                                        Backups  ...
Know Your Way Around The ProjectThursday, March 31, 2011
Know Your Way Around The Project                                 www.postgresql.org                           downloads   ...
Know Your Way Around The Project                                planet.postgresql.org                           project ne...
Know Your Way Around The Project                           www.pgfoundry.org                                              ...
Know Your Way Around The Project                            git.postgresql.org                                            ...
Know Your Way Around The Project                                 archives.postgresql.org                               mai...
Know Your Way Around The Project                            #postgresql                           irc.freenode.net        ...
Know Your Way Around The Project                           project management                                   core team ...
Get Off To A Good StartThursday, March 31, 2011
Get Off To A Good Start                           Use Package Management                              Consistent          ...
Get Off To A Good Start                           Use Package Management                              Different across sys...
Get Off To A Good Start                              Use Package Management                                  Different acr...
Get Off To A Good Start                           A Note About Versions                           Just playing? Default pa...
Get Off To A Good Start                                So What Do You Get?                                     pg_control ...
Get Off To A Good Start                                    So What Do You Get?-rw-------                 1   postgres   po...
Get Off To A Good Start                                    So What Do You Get?-rw-------                 1   postgres   po...
Get Off To A Good Start                                    So What Do You Get?-rw-------                 1   postgres   po...
Get Off To A Good Start                                    Configure Your Logging                                       $PG...
Get Off To A Good Start                           Configure Authentication                           most systems have diff...
Get Off To A Good Start                           Authentication Methods                                        TRUST     ...
Get Off To A Good Start                                            /contrib                           Trust these more tha...
Get Off To A Good Start                              Additional Supplied Modules                           “contrib”      ...
Let’s Talk About UpgradesThursday, March 31, 2011
Let’s Talk About Upgrades                                 Versioning                           First Digit (7.4.16 -> 8.2....
Let’s Talk About Upgrades                                 Versioning                           First Digit (7.4.16 -> 8.2....
Let’s Talk About Upgrades                                   Achtung!!                           Make Backups!             ...
Let’s Talk About Upgrades                           pg_dump/pg_restore                           simple                   ...
Let’s Talk About Upgrades                            the “slony” method                           not simple              ...
Let’s Talk About Upgrades                                 pg_upgrade                           in place upgrades          ...
Let’s Talk About Upgrades                             upgrading legacy databases                           < 8.2 is no lon...
Let’s Talk About Upgrades                                       is it worth it?                           any database wit...
Let’s Talk About Upgrades                                 Versioning                           First Digit (7.4.16 -> 8.2....
Let’s Talk About Upgrades                                 minor release versioning policy                           drop i...
Figure Your ConfigureThursday, March 31, 2011
Figure Your Configure                           the basics: performance                              shared_buffers        ...
Figure Your Configure                                the basics: logging                           stderr/pg_log vs. syslog...
Figure Your Configure                           other stuff worth looking at                            maintenance_work_me...
Figure Your Configure                              the basics: replication                           primary: wal_level = ‘...
Hardware For SoftwareThursday, March 31, 2011
Hardware for Software People                                    cpu and you                           connections, autovac...
Hardware for Software People                                     disk does and don’ts                           Put WAL on...
Hardware for Software People                                    disk don’ts                           NFS == Not Fully Saf...
Can You Back That Up?Thursday, March 31, 2011
Can You Back That Up?                                        types of backups                           logical (sql expor...
Can You Back That Up?                                            logical backups                           slow to create ...
Can You Back That Up?                               physical backups                           replication / failover mach...
Can You Back That Up?                           replication / failover node                             good idea, but... ...
Can You Back That Up?                                  “tarball” (pitr)                           basic idea is to copy al...
Can You Back That Up?                           filesystem snapshots (pitr)                           basic idea is to copy...
Replication, ReplicationThursday, March 31, 2011
Replication, Replication                           9.0 “Introduced” Replication In Postgres                               ...
Replication, Replication                           However, There Are Lots Of Alternatives!                               ...
Replication, Replication                              Quick Rundown: Slony                           Trigger Based        ...
Replication, Replication                            Quick Rundown: Londiste                           Trigger Based       ...
Replication, Replication                            Quick Rundown: Bucardo                           Trigger Based        ...
Replication, Replication                           Quick Rundown: Ruby Rep                           Trigger Based        ...
Replication, Replication                           Determine Your Needs: Failover                           Simple 2 Serve...
Replication, Replication                           Determine Your Needs: Switchover                              Master ->...
Replication, Replication                           Determine Your Needs: Single Read Slave                                ...
Replication, Replication                       Determine Your Needs: Multiple Read Slaves                                 ...
Replication, Replication                           A Note On Scaling Writes                           Scaling Writes Is A ...
Query Your QueriesThursday, March 31, 2011
Query Your Queries                              finding slow queries:                           log_min_duration_statement ...
Query Your Queries                               finding slow queries:                                   log analyzers     ...
Query Your Queries                           finding slow queries:                               log analyzers             ...
Query Your Queries                           fixing slow queries:                              explain analyze             ...
Query Your Queries                           fixing slow queries:                              explain analyze             ...
Query Your Queries                                    fixing slow queries:                                      indexing (b...
Query Your Queries                                    fixing slow queries:                                      indexing (p...
Query Your Queries                                      fixing slow queries:                                        indexin...
Query Your Queries                                       fixing slow queries:                                       indexin...
Query Your Queries                                        fixing slow queries:                                       indexi...
Query Your Queries                                        fixing slow queries:                                       indexi...
Query Your Queries                                        fixing slow queries:                                       indexi...
Other Stuff I Should MentionThursday, March 31, 2011
Other Stuff I Should Mention                                 pgcrypto                           cryptography              ...
Other Stuff I Should Mention                                            dblink                           pgsql -> pgsql co...
Other Stuff I Should Mention                                             *-link                           heterogenous con...
Other Stuff I Should Mention                             autonomous logging tool                           persistent logg...
Other Stuff I Should Mention                                   check_postgres                           nagios oriented mo...
Other Stuff I Should Mention                                    phpPgAdmin                           web based gui for pos...
Other Stuff I Should Mention                                        omnipitr                           advanced WAL file m...
Other Stuff I Should Mention                                            books                           PostgreSQL 9.0 Hig...
Other Stuff I Should Mention                                            books                           PostgreSQL 9.0 Hig...
Other Stuff I Should Mention                            we’re hiring :-)                             Database Operations  ...
Other Stuff I Should Mention                            we’re hiring :-)                             Database Operations  ...
El Fin                Thanks For Coming!Thursday, March 31, 2011
Upcoming SlideShare
Loading in...5
×

Pro Postgres 9

4,237

Published on

This talk gives an overview of the Postgres community, the Postgres database, and several add-on tools for Postgres. It is intended to provide a guide for someone wanting to do full time Postgres administration.

Published in: Technology

Transcript of "Pro Postgres 9"

  1. 1. Pro Postgres 9 Robert Treat, JDCon East 2011Thursday, March 31, 2011
  2. 2. Who Am I? (Why Listen To Me) PostgreSQL User Since 6.5.x DBA of High Traffic / Large PostgreSQL Instances Long Time Contributor to PostgreSQL Project Contribute / Maintain Several Open Source Projects Co-Author Beginning PHP & PostgreSQL 8 (Apress) I like to call it “Postgres” Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  3. 3. Outline What you need to know about the project Backups Getting Started Replication Upgrading Query Tuning Configuring Your Server Other stuff you should know about Hardware Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  4. 4. Know Your Way Around The ProjectThursday, March 31, 2011
  5. 5. Know Your Way Around The Project www.postgresql.org downloads security alerts documentation wiki bug reports support companies rss -> news - events - version Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  6. 6. Know Your Way Around The Project planet.postgresql.org project news helpful tips / examples community news Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  7. 7. Know Your Way Around The Project www.pgfoundry.org URI Type SkyTools modules Npgsql programs pg_bulkload resources plpgsql-debugger pagila, sample database Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  8. 8. Know Your Way Around The Project git.postgresql.org check_postgres modules edb-installers programs website code resources json type works in progress pg_top user repo’s see also github.com Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  9. 9. Know Your Way Around The Project archives.postgresql.org mailing list archives back to 1997 full text search via postgresql FTS keyword search suggestions lists for users, developers, regional, user groups Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  10. 10. Know Your Way Around The Project #postgresql irc.freenode.net real time help pg_docbot - ??help Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  11. 11. Know Your Way Around The Project project management core team committers -hackers road map web team Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  12. 12. Get Off To A Good StartThursday, March 31, 2011
  13. 13. Get Off To A Good Start Use Package Management Consistent Standardized Simple Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  14. 14. Get Off To A Good Start Use Package Management Different across systems Upgrades can be a problem Trust your packager? Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  15. 15. Get Off To A Good Start Use Package Management Different across systems Upgrades can be a problem Trust your packager? Don’t Be Afraid To Roll Your Own Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  16. 16. Get Off To A Good Start A Note About Versions Just playing? Default packages probably OK, (min 8.2) Any production level work should start on 9.0 Any project not due to launch for 6 months, 9.1 Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  17. 17. Get Off To A Good Start So What Do You Get? pg_control version number: 903 Catalog version number: 201008051 Database system identifier: 5546136951888795324 Database cluster state: in archive recovery pg_control last modified: Tue Mar 22 11:50:24 2011 Latest checkpoint location: 7C/1D000058 Prior checkpoint location: 7C/1C000058 Latest checkpoints REDO location: 7C/1D000020 Latest checkpoints TimeLineID: 1 Latest checkpoints NextXID: 0/4924 Latest checkpoints NextOID: 16608 Latest checkpoints NextMultiXactId: 1 Latest checkpoints NextMultiOffset: 0 Latest checkpoints oldestXID: 654 Latest checkpoints oldestXIDs DB: 1 pg_controldata Latest checkpoints oldestActiveXID: Time of latest checkpoint: 4924 Tue Mar 22 11:46:19 2011 Minimum recovery ending location: 7C/16000000 Backup start location: 0/0 Current wal_level setting: hot_standby Current max_connections setting: 500 Current max_prepared_xacts setting: 0 Current max_locks_per_xact setting: 64 Maximum data alignment: 8 Database block size: 8192 Blocks per segment of large relation: 131072 WAL block size: 8192 Bytes per WAL segment: 16777216 Maximum length of identifiers: 64 Maximum columns in an index: 32 Maximum size of a TOAST chunk: 1996 Date/time type storage: 64-bit integers Float4 argument passing: by value Float8 argument passing: by value Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  18. 18. Get Off To A Good Start So What Do You Get?-rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSIONdrwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 basedrwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 globaldrwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog-rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf-rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.confdrwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_logdrwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixactdrwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notifydrwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmpdrwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtransdrwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspcdrwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophaselrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog-rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf-rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts-rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid-rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  19. 19. Get Off To A Good Start So What Do You Get?-rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSIONdrwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 basedrwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 globaldrwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog-rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf-rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.confdrwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_logdrwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixactdrwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notifydrwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmpdrwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtransdrwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspcdrwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophaselrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog-rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf-rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts-rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid-rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  20. 20. Get Off To A Good Start So What Do You Get?-rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSIONdrwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 basedrwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 globaldrwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog-rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf-rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.confdrwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_logdrwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixactdrwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notifydrwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmpdrwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtransdrwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspcdrwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophaselrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog-rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf-rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts-rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid-rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  21. 21. Get Off To A Good Start Configure Your Logging $PGDATA/pg_log /var/log/postgresql when in doubt... (postgresql.conf) separate disk Logging is often overlooked, but it is the first step toward troubleshooting! Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  22. 22. Get Off To A Good Start Configure Authentication most systems have different defaults firewalls / selinux (FATAL?) rtfm (pg_hba.conf, grant, revoke) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  23. 23. Get Off To A Good Start Authentication Methods TRUST md5 IDENT Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  24. 24. Get Off To A Good Start /contrib Trust these more than your own code package dependent use different schemas (when able) pgcrypto Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  25. 25. Get Off To A Good Start Additional Supplied Modules “contrib” Trust these more than your own code package dependent use different schemas (when able) pgcrypto, pg_upgrade http://www.postgresql.org/docs/current/interactive/contrib.html Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  26. 26. Let’s Talk About UpgradesThursday, March 31, 2011
  27. 27. Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  28. 28. Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  29. 29. Let’s Talk About Upgrades Achtung!! Make Backups! Read the Release Notes! Read the Commit Logs! Test! Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  30. 30. Let’s Talk About Upgrades pg_dump/pg_restore simple -Fc is your friend dump with new version of pg_dump pitfalls: (time, hdd) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  31. 31. Let’s Talk About Upgrades the “slony” method not simple create slave on new version switchover (switch back?) pitfalls: (initial synch, compatibility) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  32. 32. Let’s Talk About Upgrades pg_upgrade in place upgrades rewrites system catalog info no way to go back (fs snapshots) first official release was 9.0 Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  33. 33. Let’s Talk About Upgrades upgrading legacy databases < 8.2 is no longer supported (upgrade now!) pg_upgrade only works on 8.3 or greater older slony can work back to 7.3ish pre-7.3 ? Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  34. 34. Let’s Talk About Upgrades is it worth it? any database with significant transaction workload (read or write) >= 8.3 (vxid) most systems will want at least 8.4 (fsm) replication is nice too though... Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  35. 35. Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  36. 36. Let’s Talk About Upgrades minor release versioning policy drop in replacement no feature changes bug fixes only (includes security fixes) sometimes involves extra steps also referred to as service packs, hot fixes, fix packs, etc... It is considered more dangerous to not upgrade! Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  37. 37. Figure Your ConfigureThursday, March 31, 2011
  38. 38. Figure Your Configure the basics: performance shared_buffers effective_cache_size work_mem checkpoint_segments Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  39. 39. Figure Your Configure the basics: logging stderr/pg_log vs. syslog/eventlog log_min_duration_statement log_line_prefix (%t, %p, %d, %u, %r) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  40. 40. Figure Your Configure other stuff worth looking at maintenance_work_mem max_prepared_transactions wal_buffers checkpoint_completion_target autovacuum_freeze_max_age Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  41. 41. Figure Your Configure the basics: replication primary: wal_level = ‘hot_standby’ primary: max_wal_senders = n slave: hot_standby = on Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  42. 42. Hardware For SoftwareThursday, March 31, 2011
  43. 43. Hardware for Software People cpu and you connections, autovacuum, bgwriter, stats collector, replication Multiple CPU’s work wonders, up to 32 processors http://tweakers.net Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  44. 44. Hardware for Software People disk does and don’ts Put WAL on it’s own disk (RAID 1) Put DATA directory on it’s own disk (RAID 10) More Spindles is Good More Controllers Even Gooder (tablespaces) Battery Backed Controllers, Write Cache Enabled Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  45. 45. Hardware for Software People disk don’ts NFS == Not Fully Safe! RAID 5 Beware disks / controllers that lie Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  46. 46. Can You Back That Up?Thursday, March 31, 2011
  47. 47. Can You Back That Up? types of backups logical (sql export) physical (files on disk) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  48. 48. Can You Back That Up? logical backups slow to create slow to restore “pure”, no system level corruption susceptible to database level corruption pg_dump is your friend pg_dumpall for globals Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  49. 49. Can You Back That Up? physical backups replication / failover machine “tarball” (pitr) filesystem snapshots (pitr) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  50. 50. Can You Back That Up? replication / failover node good idea, but... offsite? fragile? we’ll talk more in replication section :-) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  51. 51. Can You Back That Up? “tarball” (pitr) basic idea is to copy all database files and relevant xlogs use multiple machines if able use rsync if able copy the slave if able!! Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  52. 52. Can You Back That Up? filesystem snapshots (pitr) basic idea is to copy all database files and relevant xlogs use multiple machines if able use rsync if able copy the slave if able!! Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  53. 53. Replication, ReplicationThursday, March 31, 2011
  54. 54. Replication, Replication 9.0 “Introduced” Replication In Postgres Built In Solution Based on WAL replay True Binary Replication 1st Cut, Still A Little Rough Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  55. 55. Replication, Replication However, There Are Lots Of Alternatives! Most Are Trigger Based: Slony, Bucardo, Londiste, RubyRep, etc... Statement Based (PGPoolII) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  56. 56. Replication, Replication Quick Rundown: Slony Trigger Based Asynchronous Master -> (Multiple, Cascading) Slave Switchover “A Pain To Work With” Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  57. 57. Replication, Replication Quick Rundown: Londiste Trigger Based Asynchronous Master -> (Multiple) Slave Switchover “I Don’t Like Slony” Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  58. 58. Replication, Replication Quick Rundown: Bucardo Trigger Based Asynchronous Master -> (Multiple) Slave Master -> Master “Small Community” Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  59. 59. Replication, Replication Quick Rundown: Ruby Rep Trigger Based Asynchronous Master -> Slave Master -> Master “Huh?” Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  60. 60. Replication, Replication Determine Your Needs: Failover Simple 2 Server Setup? 9.0 Streaming Replication Multiple Slaves? Slony Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  61. 61. Replication, Replication Determine Your Needs: Switchover Master -> Slave Slony Londiste Master -> Master Bucardo Ruby Rep Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  62. 62. Replication, Replication Determine Your Needs: Single Read Slave Postgres Streaming Replication no changes to the slave Any of the Trigger Based Solutions Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  63. 63. Replication, Replication Determine Your Needs: Multiple Read Slaves Slony (Cascading) Bucardo (Split) Londiste (meh) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  64. 64. Replication, Replication A Note On Scaling Writes Scaling Writes Is A Hard Problem Bucardo/RubyRep Help IF CPU Bound If you are bound on write i/o, you probably need to federate your database Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  65. 65. Query Your QueriesThursday, March 31, 2011
  66. 66. Query Your Queries finding slow queries: log_min_duration_statement -1, 0, n superuser only alter user LOG: duration: 5005.273ms statement: select pg_sleep(5); Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  67. 67. Query Your Queries finding slow queries: log analyzers command line tools i/o load generate reports cpu load archive reports Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  68. 68. Query Your Queries finding slow queries: log analyzers pqa pgfouine maatkit (percona) pgsi (endpoint) Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  69. 69. Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “explain” for large queries could be it’s own talk Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  70. 70. Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “explain” for large queries could be it’s own talk http://wiki.postgresql.org/Using_EXPLAIN Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  71. 71. Query Your Queries fixing slow queries: indexing (basic) use explain to find large sequential reads use pg_stat_* tables to find numerous reads use btree (gist/gin) dual column vs. single column Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  72. 72. Query Your Queries fixing slow queries: indexing (partial) restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query create index address_ba_part_idx on address (district) where district = ‘Buenos Aires’; Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  73. 73. Query Your Queries fixing slow queries: indexing (partial) restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query create index customer_active_part_idx on customer (customer_id) where active is true; Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  74. 74. Query Your Queries fixing slow queries: indexing (functional) some people prefer to call these expressional indexes Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  75. 75. Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create unique index one_true_email_xidx on customer (lower(email)); Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  76. 76. Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create index fullname_xidx on customer ((firstname||’ ‘||lastname)); Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  77. 77. Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create index fullname_xidx on customer ((firstname||’ ‘||lastname)); Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  78. 78. Other Stuff I Should MentionThursday, March 31, 2011
  79. 79. Other Stuff I Should Mention pgcrypto cryptography /contrib (export issues) md5, sha1, blowfish, many more Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  80. 80. Other Stuff I Should Mention dblink pgsql -> pgsql connections /contrib (still under development) can have performance issues on large queries make it live in it’s own schema may be replaced by foreign data wrappers Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  81. 81. Other Stuff I Should Mention *-link heterogenous connections for postgresql db specific and db independent options any pl/u language can implement this similar performance issue as dblink dblink-tds, dbi-link, oralink, odbclink Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  82. 82. Other Stuff I Should Mention autonomous logging tool persistent logging for postgresql functions built on top of dblink make it live in it’s own schema https://labs.omniti.com/trac/pgtreats Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  83. 83. Other Stuff I Should Mention check_postgres nagios oriented monitoring script comprehensive list of items for alterting adapted for other systems, some trending http://bucardo.org/check_postgres Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  84. 84. Other Stuff I Should Mention phpPgAdmin web based gui for postgresql remote administration of multiple servers implements much of postgresql functionality http://phppgadmin.sourceforge.net/ Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  85. 85. Other Stuff I Should Mention omnipitr advanced WAL file management scripts pitr archiving, pitr restore complete filesystem based backup can create backups from a slave server https://labs.omniti.com/trac/pgtreats Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  86. 86. Other Stuff I Should Mention books PostgreSQL 9.0 High Performance, by Greg Smith PostgreSQL 9.0 Admin Cookbook, by Simon Riggs Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  87. 87. Other Stuff I Should Mention books PostgreSQL 9.0 High Performance, by Greg Smith PostgreSQL 9.0 Admin Cookbook, by Simon Riggs Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  88. 88. Other Stuff I Should Mention we’re hiring :-) Database Operations Big Systems High Volume Workloads Keggerator! Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  89. 89. Other Stuff I Should Mention we’re hiring :-) Database Operations Big Systems High Volume Workloads Keggerator! Robert Treat | Pro Postgres 9Thursday, March 31, 2011
  90. 90. El Fin Thanks For Coming!Thursday, March 31, 2011

×