(SDD409) Amazon RDS for PostgreSQL Deep Dive | AWS re:Invent 2014
•
7 likes•3,630 views
Learn the specifics of Amazon RDS for PostgreSQL's capabilities and extensions that make it powerful. This session covers database data import, performance tuning and monitoring, troubleshooting, security, and leveraging open source solutions with RDS. Throughout, this session focuses on capabilities particular to RDS for PostgreSQL.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (19)
Similar to (SDD409) Amazon RDS for PostgreSQL Deep Dive | AWS re:Invent 2014
Similar to (SDD409) Amazon RDS for PostgreSQL Deep Dive | AWS re:Invent 2014 (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
(SDD409) Amazon RDS for PostgreSQL Deep Dive | AWS re:Invent 2014
- 1. November 12, 2014 | Las Vegas, NV Grant McAlister, Amazon Web Services Greg Roberts, illumina
- 4. 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 both on fsync=0 sync commit=0 fsync=0 & sync commit=0 TransactionsperSecond 32 thread insert- fsync vs sync commit 16 segments 256 segments
- 5. 0 20 40 60 80 100 120 140 160 both on fsync=0 sync commit=0 fsync=0 & sync commit=0 Time-Seconds Bulk load 2GB of data -fsync vs sync commit 16 segments 256 segments
- 6. 29.1 28.8 26.1 25.223.9 0 5 10 15 20 25 30 35 fsync=1 & sync commit=0 fsync=0 & sync commit=0 Time-Minutes Index build on 20GB table maintenance_work_mem=16MB & checkpoint_segments=16 maintenance_work_mem=1024MB & checkpoint_segments=16 maintenance_work_mem=1024MB & checkpoint_segments=1024
- 7. 29.1 23.4 23.9 28 0 5 10 15 20 25 30 35 maintenance_work_mem Time-Minutes Index build on 20GB table 16 512 1024 4096
- 15. AZ1 AZ2 AZ3
- 16. Reads10% Reads10% Reads10% Reads10% Reads10% Reads10% Reads10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Reads10% Writes10% Reads10% Writes10% Reads10% Reads 90% Writes10% Primary Writes10% Replica1 Writes10% Replica2 Writes10% Replica3 Writes10% Replica4 1X2X3X SCALE
- 17. 1 2 3 4 5 6 7 8 1 2 4 8 16 32 Scale Number of Nodes Scale based on % Write 50% 40% 30% 20% 10%
- 18. AZ1 AZ2 AZ3
- 20. A - Foo A- Bar Source A - Foo A- Bar Replica
- 27. 0 1000 2000 3000 4000 5000 6000 7000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 TransactionsperSecond(TPS) Hours 100% Read - 20GB data db.m1.medium + 200GB standard $0.575 per hour
- 28. 0 1000 2000 3000 4000 5000 6000 7000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 TransactionsperSecond(TPS) Hours 100% Read - 20GB data db.m1.medium + 200GB standard db.m3.medium + 200G + 2000 IOPS $0.575 per hour $0.408 per hour
- 29. 0 1000 2000 3000 4000 5000 6000 7000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 TransactionsperSecond(TPS) Hours 100% Read - 20GB data db.m1.medium + 200GB standard db.m3.medium + 200G + 2000 IOPS db.m3.large + 200G + 2000 IOPS $0.575 per hour $0.408 per hour $0.508 per hour
- 30. 0 1000 2000 3000 4000 5000 6000 7000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 TransactionsperSecond(TPS) Hours 100% Read - 20GB data db.m1.medium + 200GB standard db.m3.medium + 200G + 2000 IOPS db.m3.large + 200G + 2000 IOPS db.t2.medium + 200GB gp2 $0.105 per hour $0.575 per hour $0.408 per hour $0.508 per hour
- 31. 0 1000 2000 3000 4000 5000 6000 7000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 TransactionsperSecond(TPS) Hours 100% Read - 20GB data db.m1.medium + 200GB standard db.m3.medium + 200G + 2000 IOPS db.m3.large + 200G + 2000 IOPS db.t2.medium + 200GB gp2 db.t2.medium + 1TB gp2 $0.105 per hour $0.575 per hour $0.233 per hour $0.408 per hour $0.508 per hour
- 35. Custom docker job scheduler – Simple design – needed fast and reliable ACID transactions – Semi-structured and relational data API service SpaceDock Docker BSFS - FUSE Docker registry Learn more at: developer.basespace.illumina.com Mission control mission.basespace.illumina.com Elastic Load Balancing API Server(s) on EC2 Mission Control Service Amazon RDS PostgreSQL Amazon RDS DB (Multi-AZ) Locked down Security Group On-Demand Linux EC2 instances with mounted volumes Schedules Job Native App AMI BSFS - FUSE drive SpaceDock App Inp ut Output App Docker Image Cancels on failure Amazon S3 Data Transfer Developer Machines/ External Instances Can Receive Jobs just like on-demand instance Amazon Route 53 DNS
- 38. http://pgtune.leopard.in.ua/ Config Info Default New Max_Connections Is based on memory and not CPU. Generally lower this to match max settings of app pool. InstanceMemory/125 82880 =2836 1000 Shared_Buffers Some may consider conservative, but depends on app. 25% - 40% is ideal in 9.3. InstanceMemory/327 68=8 GB Same Work_Mem Too conservative especially if you lower max_connections Engine Default= 1MB 3MB Maintenance_Work_Mem Red flag, way too low. This determines vacuum and index performance. Recommend ~1/16 system memory. Engine Default = 16MB 1.9 GB Effective_Cache_Size Recommended range is 50-75% system memory. InstanceMemory/163 84 = 17 GB Same
- 39. +50% 0 2000 4000 6000 8000 db.m2.2xlarge db.m3.2xlarge Total Time - Seconds (smaller better)