HA+DRBD+Postgres - PostgresWest '08
Upcoming SlideShare
Loading in...5
×
 

HA+DRBD+Postgres - PostgresWest '08

on

  • 16,715 views

These are the slides for the HA+DRBD+Postgres talk I gave at PostgresWest '08.

These are the slides for the HA+DRBD+Postgres talk I gave at PostgresWest '08.

Statistics

Views

Total Views
16,715
Views on SlideShare
16,597
Embed Views
118

Actions

Likes
9
Downloads
352
Comments
0

8 Embeds 118

http://www.slideshare.net 94
http://www.linkedin.com 8
http://www.techgig.com 6
https://www.linkedin.com 6
http://webcache.googleusercontent.com 1
http://www.m.techgig.com 1
http://115.112.206.131 1
http://127.0.0.1 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

HA+DRBD+Postgres - PostgresWest '08 HA+DRBD+Postgres - PostgresWest '08 Presentation Transcript

  • Linux‐HA
+
DRBD
+
PostgreSQL
 Jesse
Young

  • Target
Audience
 •  System
Administrators
 •  DBAs
looking
into
replicaBon

  • Goals
 •  Inspire
more
replicaBon
discussions
 •  Inform
System
Admins/DBAs
 •  Describe
components
 •  Simple
Setup
 •  DemonstraBon
 •  Performance
Tips

  • Intro
 •  Zonar
Systems,
Inc.
 –  45,000
vehicles
sending
data
in
24/7
and
growing
 –  15
database
servers
 –  5
TB
data,
growing
~25GB
daily

 –  PostgreSQL
+
PostGIS

  • Why
use
Linux‐HA
+
DRBD?
 •  Simple
replicaBon
 •  GPS
data
coming
into
the
system
24/7
 •  MiBgate
problems
from
server
hardware
 •  Users
and
IntegraBon
soYware
access

  • Linux‐HA
 •  “Provide a high availability (clustering)  solu6on for Linux which promotes reliability,  availability, and serviceability (RAS) through a  community development effort.”
 •  System
Admins:
You
can
conBnue
to
(half)
 sleep,
even
if
a
server
dies.

  • DRBD
 •  Distributed
Replicated
Block
Device
 •  Mirrored
storage
across
servers
 Image
from
h_p://www.drbd.org

  • PostgreSQL
 •  Database
engine
we
all
know
and
love

  • Setup
 •  Dual‐proc
Quad‐core
Intel
Xeon
@
2.66
GHz
 •  16GB
RAM
 •  2x
74
GB
SCSI
@
15kRPM
(RAID
1
‐
OS)
 •  6x
300
GB
SCSI
@
10kRPM
(RAID
10
–
DRBD)
 •  LSI
Megaraid
–
2
Channel,
512
MB
Cache
w/
 BBU

  • Quick
Install
 •  Versions
 –  heartbeat‐2.0.8
 –  drbd‐8.0.5
 –  PostgreSQL
–
Any
version

  • Install
DRBD
 •  Compile
from
source
 or
 •  yum
install
drbd
kmod‐drbd

  • Configure
DRBD
 resource
postgres
{

 on
node‐001
{
 




device
/dev/drbd0;
 




disk


/dev/sdb1;
 




address
10.40.0.1:7788;
 




meta‐disk

internal;
 

}
 

on
node‐002
{
 




device
/dev/drbd0;
 




disk


/dev/sdb1;
 




address
10.40.0.2:7788;
 




meta‐disk

internal;
 

}
 }

  • Begin
DRBD
 •  Each
node:
 –  modprobe
drbd
 –  drbdadm
create‐md
postgres
 –  drbadm
up
all

  • Create
ParBBon
on
DRBD
Device
 •  Primary
node:
 –  drbdadm
‐‐
‐‐overwrite‐data‐of‐peer
primary
all
 –  mke2fs
‐j
/dev/drbd0

 –  mount
device
 –  Edit
/etc/init.d/postgresql
 •  PGDATA
to
DRBD
device
 –  initdb
or
sync/copy
exisBng
database

  • DRBD
status
 •  cat
/proc/drbd

 version:
8.0.5
(api:86/proto:86)
 SVN
Revision:
3011
build
by
jesse@node‐001.zonarsystems.net,
2007‐09‐02
23:16:51
 
0:
cs:Connected
st:Primary/Secondary
ds:UpToDate/UpToDate
C
r‐‐‐
 



ns:191681144
nr:108784
dw:191798660
dr:1228217841
al:962834152
bm:9054
lo:0
 pe:0
ua:0
ap:0
 
resync:
used:0/31
hits:34633
misses:4367
starving:0
dirty:0
changed:4367
 
act_log:
used:0/127
hits:11433119293
misses:1051777270
starving:11558178
 dirty:81149096
changed:962834152

  • Install
Linux‐HA
 •  Compile
from
source
 or
 •  yum
install
heartbeat

  • Configure
Linux‐HA
 •  ha.cf
 •  haresources
 •  authkeys

  • ha.cf
 deadBme
120
 auto_failback
off
 udpport
694
 ucast
eth1
10.30.0.10
 node



node‐001.zonarsystems.net
 node



node‐002.zonarsystems.net

  • haresources
 cluster‐001.zonarsystems.net

IPaddr::10.30.0.101/24
drbddisk::postgres

 




























Filesystem::/dev/drbd0::/mnt/rod1::ext3

 




























postgresql

 MailTo::jesse@zonarsystems.com::'Zonar_Cluster‐001_Switch'

  • authkeys
 auth
1
 1
sha1
f93e5e562d430dae92b7fd7b272c13bb0c5de0e2

  • Start
Linux‐HA
 •  /etc/init.d/heartbeat
start

  • DEMO

  • Problems?
 •  Secondary
server
always
stand‐by
 •  Failed
node
can
cause
slow
downs
 •  Kernel
module

  • Performance?
 •  dd
if=/dev/zero
of=/mnt/drbd/pgwest/test
bs=8192
 count=2000000
 •  DRBD
device
 –  ~85
MB/s
 •  Disk
device
 –  121MB/s
 30%
write performance
degradaBon

  • Problems?

  • Performance
Improvements

  • Ethernet
 •  Direct
cross
over
cable
 •  GigE,
10
GigE
 •  1500
MTU
vs
9000
MTU

  • RAID
Cards
 •  Capable
of
WRITEBACK
 •  Write
caching
 •  Ba_ery
backup
unit
(BBU)

  • Writeback
VS.
Writethrough
 100
 •  Writeback
 90
 80
 –  80MB/s
–
94MB/s
 70
 60
 50
 40
 •  Writethrough
 30
 –  61MB/s
–
71MB/s
 20
 10
 0
 Run
 Run
 Run
 Run
 Run
 Run
 Run
 Run
 1
 3
 5
 7
 9
 11
 13
 15

  • DRBD
Protocols
 •  Protocol
A
 –  DRBD
waits
for
local
disk
and
local
TCP
send
buffer
 •  Protocol
B
 –  DRBD
waits
for
local
disk
and
remote
buffer
cache
 •  Protocol
C
 –  DRBD
waits
for
both
local
and
remote
disk
 –  SAFEST

  • YMMV
 •  Remember
to
test
yourself
 •  Fedora/e1000
network
stack
issues
 •  DRBD
is
soYware
too,
bugs
come
up

  • Other

Uses
 HA‐Linux
+
DRBD
+
???
 •  Mail
 •  NFS
 •  MySQL

  • QuesBons?

  • Links
 •  Linux‐HA:
h_p://www.linux‐ha.org
 •  DRBD:
h_p://www.drbd.org

  • Demo
(backup
plan)

  • Host:
c6a

  • Host:
c6a

  • Host:
c6a

  • Host:
c6a

  • Host:
c6a

  • Host:
c6b

  • Host:
c6b

  • Host:
c6a