Learn the specifics of Amazon RDS for PostgreSQL’s capabilities and extensions that make it powerful. This session begins with a brief overview of the RDS PostgreSQL service, how it provides High Availability & Durability and will then deep dive into the new features that we have released since re:Invent 2014, including major version upgrade and newly added PostgreSQL extensions to RDS PostgreSQL. During the session, we will also discuss lessons learned running a large fleet of PostgreSQL instances, including specific recommendations. In addition we will present benchmarking results looking at differences between the 9.3, 9.4 and 9.5 releases.

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Grant McAlister – Senior Principal Engineer - RDS
October 2015
DAT402
Amazon RDS for PostgreSQL
Lessons Learned and Deep Dive on New Features

2. Major version upgrade
Coming
Soon
Prod
9.3
Prod
9.4
pg_upgrade
Backup Backup
No PITR
Test
9.3
Test
9.4
pg_upgrade
Restore to a test instance
Application
Testing

3. What’s new in storage
6TB storage
• PIOPS has 30K IOPS max
• GP2 increase storage above 3TB = increase throughput & IOPS
Encryption at rest
• Uses the AWS Key Management Service (KMS) part of AWS
Identity and Access Management (IAM)
• Your own key
• Use a default one
• Includes all data files, log files, log backups, and snapshots

4. 0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
2 Threads 4 Threads 8 Threads 16 Threads 32 Threads 64 Threads
TransactionsPerSecond(TPS)
PG Bench - Read Only - In Memory
Regular
Encrypted
Encryption at rest overhead
No measureable overhead

5. 0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
2 Threads 4 Threads 8 Threads 16 Threads 32 Threads 64 Threads
TransactionsPerSecond(TPS)
PG Bench - Read & Write
Regular
Encrypted
Encryption at rest overhead
5 to 10% Overhead on heavy write

6. Version updates
RDS now supports
• 9.3.6 – Fix for RDS Bug – RESET ALL
• 9.3.9 (Default)
• 9.4.1 and 9.4.4 (Default)
• JSONB
• GIN Index Improvements
• pg_prewarm extension
• New PLV8 & PostGIS versions

7. Operating System (OS) metrics
5 second granularity
Coming
SooncpuUtilization
• guest
• irq
• system
• wait
• idl:
• user
• total
• steal
• nice
diskIO
• writeKbPS
• readIOsPS
• await
• readKbPS
• rrqmPS
• util
• avgQueueLen
• tps
• readKb
• writeKb
• avgReqSz
• wrqmPS
• writeIOsPS
memory
• writeback
• cached
• free
• inactive
• dirty
• mapped
• active
• total
• slab
• buffers
• pageTable
swap
• cached
• total
• free
tasks
• sleeping
• zombie
• running
• stopped
• total
• blocked
fileSys
• used
• usedFiles
• usedFilePercent
• maxFiles
• total
• usedPercent
loadAverageMinute
• fifteen
• five
• one
uptime
processList
• name
• cpuTime
• parentID
• memoryUsedPct
• cpuUsedPct
• id
• rss
• vss

8. OS metrics

9. Data movement

10. Move data to the same or different database engine
Keep your apps running during the migration
Start your first migration in 10 minutes or less
Replicate within, to, or from AWS EC2 or RDS
AWS
Database Migration
Service

11. Customer
Premises
Application Users
EC2
or
RDS
Internet
VPN
Start a replication instance
Connect to source and target databases
Select tables, schemas, or databases
Let the AWS Database Migration
Service create tables and load data
Uses change data capture to keep
them in sync
Switch applications over to the target
at your convenience
Keep your apps running during the migration
AWS Database
Migration Service

12. AWS Database Migration Service - PostgreSQL
• Source - on premises or Amazon EC2 PostgreSQL (9.4)
• Destination can be EC2 or RDS
• Initial bulk copy via consistent select
• Uses PostgreSQL logical replication support to provide
change data capture
http://aws.amazon.com/rds/DatabaseMigrationService/preview

13. Loading data
• Disable backups – backup_retention=0
• Disable Multi-AZ & autovacuum
• pg_dump –Fc (compressed) pg_restore –j (parallel)
• Increase maintenance_work_mem
• Increase checkpoint_segments & checkpoint_timeout
• Disable FSYNC
• Disable synchronous_commit

14. 0
2000
4000
6000
8000
10000
12000
14000
16000
18000
both on fsync=0 sync commit=0 fsync=0 & sync commit=0
TransactionsperSecond
32 thread insert- fsync vs sync commit
16 segments 256 segments

15. 0
20
40
60
80
100
120
140
160
both on fsync=0 sync commit=0 fsync=0 & sync commit=0
Time-Seconds
Bulk load 2GB of data -fsync vs sync commit
16 segments 256 segments

16. 29.1 28.8
26.1
25.223.9
0
5
10
15
20
25
30
35
fsync=1 & sync commit=0 fsync=0 & sync commit=0
Time-Minutes
Index build on 20GB table
maintenance_work_mem=16MB &
checkpoint_segments=16
maintenance_work_mem=1024MB &
checkpoint_segments=16
maintenance_work_mem=1024MB &
checkpoint_segments=1024

17. Vacuuming – 100% read-only workload

18. Vacuum parameters
Will auto vacuum when
• autovacuum_vacuum_threshold +
autovacuum_vacuum_scale_factor * pgclass.reltuples
How hard auto vacuum works
• autovacuum_max_workers
• autovacuum_nap_time
• autovacuum_cost_limit
• autovacuum_cost_delay

19. postgres_fdw + Amazon Redshift

20. session_replication_role
Table
Foo
Trigger
Table
Foo
Trigger
DB1 DB2
insert

21. Scale and availability

22. shared_buffers parameter
244GB RAM
PG processes
shared_buffers
Linux
pagecache
select of data – check for buffer in shared_buffers
if not in shared_buffers load from pagecache/disk
EBS
1/4
shared_buffers = working set size

23. 0
2,000
4,000
6,000
8,000
10,000
12,000
3% 6% 13% 25% 50% 75%
transactionspersecond(TPS)
shared_buffers as a percentage of system memory
pgbench write workload on r3.8xlarge
working set = 10% of memory
25 threads
50 threads
100 threads
200 threads
400 threads
800 threads

24. 0
2,000
4,000
6,000
8,000
10,000
12,000
13% 25% 50% 75%
transactionspersecond(TPS)
shared_buffers as a percentage of system memory
pgbench write workload on r3.8xlarge
working set = 50% of memory
25 threads
50 threads
100 threads
200 threads
400 threads
800 threads

25. Availability – Read and Write – Multi-AZ
Physical
Synchronous
Replication
AZ1 AZ2
DNS
cname update
Primary Update

26. Read Replicas = Availability
Sync
Replication
Multi-AZ
Async Replication

27. Read Replica promotion
AZ1 AZ2 AZ3

28. Read Replicas = Scale
AZ1 AZ2 AZ3

29. Replication parameters
wal_keep_segments
xlog1
xlog2
xlog3
xlog99
xlog1
xlog1

30. pg_stat_replication
benchdb=> select * from pg_stat_replication;
-[ RECORD 1 ]----+--------------------------------------------
pid | 40385
usesysid | 16388
usename | rdsrepladmin
application_name | walreceiver
client_addr | 10.22.132.253
client_hostname | ip-10-22-132-253.us-west-2.compute.internal
client_port | 22825
backend_start | 2014-10-29 21:44:58.080324+00
state | streaming
sent_location | 98/7A000900
write_location | 98/7A000900
flush_location | 98/7A000900
replay_location | 98/7A000900
sync_priority | 0
sync_state | async

31. Replication parameters – continued
vacuum_defer_cleanup_age
max_standby_archive_delay
max_standby_streaming_delay
hot_standby_feedback
A - Foo
A- Bar
Source
A - Foo
A- Bar
Replica

32. vacuum_defer_cleanup_age
on primary
default is 0
# of transactions
Table T1
t1 – foo, bar
t2 – foo, car
t3 – foo, dar
t4 – foo, ear
t5 – foo, far
t6 – foo, gar
t1 – foo, bar
t2 – foo, car
t3 – foo, dar
t4 – foo, ear
t5 – foo, far

33. max_standby_archive/streaming_delay
xlog1
Not all sessions will see the max delay

34. hot_standby_feedback
xlog1
select * from t1select * from t1

35. pg_stat_database_conflicts
benchdb=> select * from pg_stat_database_conflicts;
datid | datname | confl_tablespace | confl_lock | confl_snapshot | confl_bufferpin | confl_deadlock
-------+-----------+------------------+------------+----------------+-----------------+----------------
12891 | template0 | 0 | 0 | 0 | 0 | 0
16384 | rdsadmin | 0 | 0 | 0 | 0 | 0
1 | template1 | 0 | 0 | 0 | 0 | 0
12896 | postgres | 0 | 0 | 0 | 0 | 0
16394 | benchdb | 0 | 0 | 0 | 0 | 0
32810 | bench2 | 0 | 0 | 1 | 0 | 0

36. pg_stat_statements
Change parameter shared_preload_libraries=pg_stat_statements
=>create extenstion pg_stats_statements
=>select query, calls, total_time, rows, shared_blks_read from
pg_stat_statements where total_time > 100 and query like '%usertable%';
query | calls | total_time | rows | shared_blks_read
-------------------------------------------------------------------------------------------+----------+------------------+------------+-----------------
SELECT * FROM usertable WHERE YCSB_KEY = $1 | 71356782 | 8629119.24887683 | 71356780 | 28779668
SELECT * FROM usertable WHERE YCSB_KEY >= $1 LIMIT ? | 12068394 | 62530609.930002 | 1206839246 | 171093346
UPDATE usertable SET FIELD1=$1 WHERE YCSB_KEY = $2 | 7048967 | 35813107.3580354 | 7048967 | 3825857
analyze usertable; | 1 | 2129.84 | 0 | 15679
SELECT * FROM usertable WHERE YCSB_KEY >= $1 AND md5(YCSB_KEY) = md5(YCSB_KEY) LIMIT ? | 15441280 | 39356905.8080029 | 1544127640 | 230668106

37. Burst mode: GP2 and T2
T2 – Amazon EC2 instance with burst capability
• Base performance + burst
• Earn credits per hour when below base performance
• Can store up to 24 hours worth of credits
• Amazon CloudWatch metrics to see credits and usage
GP2 – SSD-based Amazon EBS storage
• 3 IOPS per GB base performance
• Earn credits when usage below base
• Burst to 3000+ IOPS

38. T2 – CPU credits

39. Burst mode: what’s new
db.t2.large
• 60 CPU Initial Credit
• 36 CPU Credit earned per hour
• Base Performance – 60%
• 8 GB RAM
• Increased IO bandwidth
• Encryption at rest support

40. 0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
TransactionsperSecond(TPS)
Hours
100% Read - 20GB data
db.m1.medium + 200GB standard
Burst mode vs. classic vs. Provisioned IOPS
$0.58 per hour

41. 0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
TransactionsperSecond(TPS)
Hours
100% Read - 20GB data
db.m1.medium + 200GB standard
db.m3.medium + 200G + 2000 IOPS
Burst mode vs. classic vs. Provisioned IOPS
$0.58 per hour
$0.40 per hour

42. 0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
TransactionsperSecond(TPS)
Hours
100% Read - 20GB data
db.m1.medium + 200GB standard
db.m3.medium + 200G + 2000 IOPS
db.m3.large + 200G + 2000 IOPS
Burst mode vs. classic vs. Provisioned IOPS
$0.58 per hour
$0.40 per hour
$0.50 per hour

43. 0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
TransactionsperSecond(TPS)
Hours
100% Read - 20GB data
db.m1.medium + 200GB standard
db.m3.medium + 200G + 2000 IOPS
db.m3.large + 200G + 2000 IOPS
db.t2.medium + 200GB gp2
Burst mode vs. Classic vs. Provisioned IOPS
$0.10 per hour
$0.58 per hour
$0.40 per hour
$0.50 per hour

44. 0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
TransactionsperSecond(TPS)
Hours
100% Read - 20GB data
db.m1.medium + 200GB standard
db.m3.medium + 200G + 2000 IOPS
db.m3.large + 200G + 2000 IOPS
db.t2.medium + 200GB gp2
db.t2.medium + 1TB gp2
Burst mode vs. Classic vs. Provisioned IOPS
$0.10 per hour
$0.58 per hour
$0.23 per hour
$0.40 per hour
$0.50 per hour

45. 0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
TransactionsperSecond(TPS)
Hours
100% Read - 20GB data
db.m1.medium + 200GB standard
db.m3.medium + 200G + 2000 IOPS
db.m3.large + 200G + 2000 IOPS
db.t2.medium + 200GB gp2
db.t2.medium + 1TB gp2
db.t2.large + 1TB gp2
Burst mode vs. Classic vs. Provisioned IOPS
$0.10 per hour
$0.58 per hour
$0.23 per hour
$0.40 per hour
$0.50 per hour
$0.30 per hour

46. Thank you!

47. Remember to complete
your evaluations!