High Availability and PostgreSQL
Fujitsu Coffee Morning — July 28th, 2006
                 Gavin Sherry
              gavi...
What is high availability?
A system to reduce application down time
Usually consists of
  Master server(s)
  Slave server(...
What high availability is not
... a mechanism to increase performance
       Usually HA comes at the price of performance
...
Choosing a HA solution
How much down time can you afford?
How much data (how many seconds, minutes) can you
afford to lose...
Failure detection
‘heartbeat’ / ‘ping’
Can be performed via: serial, fibre, Ethernet. . .
  Red Hat Cluster Suite, Linux-HA...
Switch over
IP take over
   Managed or automatic?
Application reconfiguration
Middleware reconfiguration
Shoot the master in...
Data availability mechanisms
Historically seen to be the most complex part of
database HA
   Replicating data is easy, doi...
Shared storage
Master and slave(s) connect to shared disk/SAN
Pros
   Little to no performance impact
   Simple design
   ...
Data replication: Slony I
Trigger-based row level replication
Data is pulled by slave server(s)
Pros
   Implemented by Pos...
Data replication: file system
Block based OS file system copy on write
DRBD on Linux
Pros
   Theoretically, no data loss
   ...
Data replication: log shipping
Point in time recovery (PITR)
Write ahead logs (WAL) are shipped to slave and
replayed
Pros...
SQL proxy: PG Cluster
Write statements are sent to all nodes via a proxy
Pros
   Online slaves
   Simplified design
Cons
  ...
SQL proxy: PgPool
Much like PG Cluster
Other cons
   One slave




                             High Availability and Post...
SQL proxy: Sequoia, p/cluster
J2EE/JDBC two phase commit statement based
replication
Derived from c-JDBC by Continuent
Pro...
Testing
Test your HA solution(s)
... but thoroughness is hard




                               High Availability and Pos...
The future
Lots of next generation HA for PostgreSQL on the
horizon
   PgPool 2.0
   PgCluster 2.0 — shared disk, shared m...
Cautionary notes
HA solutions can still fail
SANs can blowup
Split brain
Replication software can fail — corrupt data or s...
References
Lifekeeper, by SteelEye - http://www.steeleye.com
Linux-HA - http://www.linux-ha.org/
Red Hat Cluster Suite -
h...
Upcoming SlideShare
Loading in …5
×

Creating customized openSUSE versions with SUSE Studio

1,555
-1

Published on

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,555
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Creating customized openSUSE versions with SUSE Studio

  1. 1. High Availability and PostgreSQL Fujitsu Coffee Morning — July 28th, 2006 Gavin Sherry gavin@alcove.com.au High Availability and PostgreSQL – p.
  2. 2. What is high availability? A system to reduce application down time Usually consists of Master server(s) Slave server(s) Software to detect the failure of a master Software to promote a slave to master status Software or hardware to ensure data consistency between the master(s) and slave(s) Sometimes technologies provide HA incidentally (more later) High Availability and PostgreSQL – p.
  3. 3. What high availability is not ... a mechanism to increase performance Usually HA comes at the price of performance ... a way to simplify your network ... cheap ... easy to implement High Availability and PostgreSQL – p.
  4. 4. Choosing a HA solution How much down time can you afford? How much data (how many seconds, minutes) can you afford to lose? Do you need a geographically dispersed solution? Are slaves online or offline? Are online upgrades possible? High Availability and PostgreSQL – p.
  5. 5. Failure detection ‘heartbeat’ / ‘ping’ Can be performed via: serial, fibre, Ethernet. . . Red Hat Cluster Suite, Linux-HA More complex OS level interconnect HACMP (AIX), Lifekeeper (Linux), Sun Cluster (Solaris) High Availability and PostgreSQL – p.
  6. 6. Switch over IP take over Managed or automatic? Application reconfiguration Middleware reconfiguration Shoot the master in the head High Availability and PostgreSQL – p.
  7. 7. Data availability mechanisms Historically seen to be the most complex part of database HA Replicating data is easy, doing it with transactional semantics is hard Minimising performance impact is also hard The slave server(s) need access to the data How far out of date can the slave(s) be? High Availability and PostgreSQL – p.
  8. 8. Shared storage Master and slave(s) connect to shared disk/SAN Pros Little to no performance impact Simple design May utilise existing infrastructure Theoretically no data loss in failure scenario Cons SANs can fail — especially cheap ones No off site capability Upgrades can be painful Slave(s) are offline Homogenous cluster High Availability and PostgreSQL – p.
  9. 9. Data replication: Slony I Trigger-based row level replication Data is pulled by slave server(s) Pros Implemented by PostgreSQL developers Online upgrades are a feature Data is replicated with transaction semantics Online slaves Off site capable Heterogeneous cluster Cons Performance impact: 1% to 5% Complex administration High Availability and PostgreSQL – p.
  10. 10. Data replication: file system Block based OS file system copy on write DRBD on Linux Pros Theoretically, no data loss Simplified implementation Cons Significant performance impact — more than 10% Off site will increase performance impact No online upgrade Offline slaves Homogenous cluster High Availability and PostgreSQL – p. 1
  11. 11. Data replication: log shipping Point in time recovery (PITR) Write ahead logs (WAL) are shipped to slave and replayed Pros <1% performance impact Simplified design Off site capable Cons Offline slaves Minimising data loss is tricky and error prone Lacks tools No online upgrade High Availability and PostgreSQL – p. 1
  12. 12. SQL proxy: PG Cluster Write statements are sent to all nodes via a proxy Pros Online slaves Simplified design Cons Data inconsistencies across servers Not two-phase commit — query may fail a server Does not handle non-deterministic queries INSERT INTO SALES VALUES(1001, current_timestamp); High Availability and PostgreSQL – p. 1
  13. 13. SQL proxy: PgPool Much like PG Cluster Other cons One slave High Availability and PostgreSQL – p. 1
  14. 14. SQL proxy: Sequoia, p/cluster J2EE/JDBC two phase commit statement based replication Derived from c-JDBC by Continuent Pros Data consistency with 2PC p/cluster does complete HA, pretty tools Simplified design Online slaves Cons Only supports JDBC (other interfaces planned) Off site may have a huge performance impact Performance impact would be non-trivial, especially with more slaves High Availability and PostgreSQL – p. 1
  15. 15. Testing Test your HA solution(s) ... but thoroughness is hard High Availability and PostgreSQL – p. 1
  16. 16. The future Lots of next generation HA for PostgreSQL on the horizon PgPool 2.0 PgCluster 2.0 — shared disk, shared memory Postgres-R/Slony-2 High Availability and PostgreSQL – p. 1
  17. 17. Cautionary notes HA solutions can still fail SANs can blowup Split brain Replication software can fail — corrupt data or stop replicating Most common issue 1. Network/software/hardware blip 2. Heart beat to fails 3. Switch over to takes place 4. Something goes wrong Network reconfiguration doesn’t work Slave server is misconfigured Slave server doesn’t kill master properly High Availability and PostgreSQL – p. 1
  18. 18. References Lifekeeper, by SteelEye - http://www.steeleye.com Linux-HA - http://www.linux-ha.org/ Red Hat Cluster Suite - http://www.redhat.com/solutions/clustersuite/ Slony - http://www.slony.info DRBD - http://www.drbd.org PG Cluster - http://pgcluster.projects.postgresql.org PgPool - http://pgpool.projects.postgresql.org/ Continuent - http://www.continuent.com Postgres-R(8) - http://www.postgres-r.org High Availability and PostgreSQL – p. 1
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×