Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Best Practices for Running PostgreSQL on AWS - DAT314 - re:Invent 2017

3,440 views

Published on

PostgreSQL is an open source database growing in popularity because of its rich features, vibrant community, and compatibility with commercial databases. Learn about ways to run PostgreSQL on AWS including self-managed, and the managed database services from AWS: Amazon Relational Database Service (Amazon RDS) and the Amazon Aurora PostgreSQL-compatible Edition. This talk covers key Amazon RDS for PostgreSQL functionality, availability, and management. We also review general guidelines for common user operations and activities such as migration, tuning, and monitoring for their RDS for PostgreSQL instances.

  • Be the first to comment

Best Practices for Running PostgreSQL on AWS - DAT314 - re:Invent 2017

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Best Practices for Running PostgreSQL on AWS Jignesh Shah S r . P r o d u c t M a n a g e r , A m a z o n R D S P o s t g r e S Q L AWS re:INVENT D e c e m b e r 1 , 2 0 1 7
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PostgreSQL—Open Source Database • Open source database • In active development for 20 years • Owned by a foundation, not a single company • Permissive innovation-friendly open source license • High performance out of the box • Object-oriented and ANSI-SQL:2008 compatible • Most geospatial features of any open source database • Supports stored procedures in 12 languages (Java, Perl, Python, Ruby, Tcl, C/C++, its own Oracle-like PL/pgSQL, etc.) • Most Oracle-compatible open source databases • Highest AWS Schema Conversion Tool automatic conversion rates are from Oracle to PostgreSQL
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PostgreSQL—Key Features • SQL support • Clustered indexes • Flexible datatypes • Parallel queries • Parallel DML • Window and custom functions • Transaction control in Stored Procedure • Custom languages • Extensions
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PostgreSQL Deployment Options On-Premises Hosted Managed EC2 DB Services
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. On-Premise PostgreSQL Deployment Power, HVAC, net Rack and stack Server maintenance OS patches DB s/w patches Database backups Scaling High availability DB s/w installs OS installation you App optimization
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Hosting PostgreSQL on Amazon EC2 Power, HVAC, net Rack and stack Server maintenance OS patches DB s/w patches Database backups Scaling High availability DB s/w installs OS installation you App optimization
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Managed PostgreSQL on AWS Power, HVAC, net Rack and stack Server maintenance OS patches DB s/w patches Database backups App optimization High availability DB s/w installs OS installation you Scaling
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Security and Compliance • Network isolation • Database instance IP firewall protection • AWS IAM-based resource-level permission controls • Encryption at rest using AWS KMS or Oracle/Microsoft TDE • SSL protection for data in transit • Assurance programs for finance, healthcare, government, and more
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Virtual Private Cloud (Amazon VPC) Securely control network configuration Availability Zone AWS Region 10.1.0.0/16 10.1.1.0/24 Manage connectivity AWS Direct Connect VPN connection VPC peering Internet gateway Routing rules
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IAM-Governed Access Use AWS Identity and Access Management (IAM) to control who can perform actions on Amazon RDS Users and DBAApplications DBA and Ops Your database RDS Controlled with IAMControlled with database grants
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. At-Rest Encryption Two-tiered key hierarchy using envelope encryption • Unique data key encrypts customer data • AWS KMS master keys encrypt data keys • Available for all RDS engines Benefits: • Limits risk of compromised data key • Better performance for encrypting large data • Easier to manage small number of master keys than millions of data keys • Centralized access and audit of key activity Leveraging AWS Key Management Service (AWS KMS) Data key 1 Data key 2 Data key 3 Data key 4 Customer master key(s) Amazon RDS instance 3 Amazon RDS instance 2 Amazon RDS instance 1
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Compliance Singapore MTCS 27001/9001 27017/27018
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Effortless Scaling Scale compute/memory vertically up or down Handle higher load to grow over time Lower usage to control costs 6 TB
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Backup and Disaster Recovery
  15. 15. Availability—Read and Write—Multi-AZ Physical Synchronous Replication AZ1 AZ2 DNS cname update Primary Update
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Read Replicas • Relieve pressure on your master node with additional read capacity • Bring data close to your applications in different regions • Promote a read replica to a master for faster recovery in the event of disaster
  17. 17. Read Replicas = Availability Sync Replication Multi-AZ Async Replication
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cross Region Replicas—Reduce Latency AZ1 AZ2 AZ1 Async Replication US-EAST-1 EU-WEST-1
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cross Region Replicas—DR & Moves AZ1 AZ2 AZ1 Async Replication US-EAST-1 EU-WEST-1 EU-WEST-1 AZ2
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Major Version Upgrade Prod 9.5 Prod 9.6 pg_upgrade Backup Backup No PITR Test 9.5 Test 9.6 pg_upgrade Restore to a test instance Application Testing
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Upgrade with Minimum Downtime Using DMS Start a replication instance Connect to source and target databases Select tables, schemas, or databases RDS PostgreSQL 9.4 Application Users RDS PostgreSQL 9.6 Let the AWS Database Migration Service create tables and load data Uses change data capture to keep them in sync Switch applications over to the target at your convenience AWS Database Migration Service
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Logical Replication Support • Supported with 9.6.1+, 9.5.4+ and 9.4.9+ • Set rds.logical_replication parameter to 1 • As user who has rds_replication and rds_superuser role SELECT * FROM pg_create_logical_replication_slot('test_slot', 'test_decoding'); pg_recvlogical -d postgres --slot test_slot -U master --host $rds_hostname -f - --start • Support for event triggers RDS Postgres RDS Postgres Logical Replica Redshift EC2 Postgres On Premise Postgres DMS RDS/Aurora PostgreSQL Custom Logical Handler NoSQL DB
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon RDS for PostgreSQL Supporting Latest Minor Releases • 9.6.5 • 9.5.9 • 9.4.14 • 9.3.19 Next Major release—PostgreSQL 10
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Notable Features Support HugePages for RDS PostgreSQL • Provides performance benefit for large instances Stop and start of RDS PostgreSQL instances New regions supported for RDS PostgreSQL • China • Brazil Support encryption at rest for • T2 small/medium • Cross-region Read Replicas Flexible RI
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PostgreSQL Extensions/Modules Supported rds-postgres-extensions-request@amazon.com 9.3 Original—32 9.3 Current—35 9.4 Current—39 9.5 Current—46 Future—??? 9.6 Current—57
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. New PostgreSQL Extensions Supported Extensions Description pgrouting Provides geospatial routing functionality for PostGIS postgresql-hll HyperLogLog data type support decoder_raw Output plugin to generates raw queries for logical replication changes pg_repack Remove bloat from tables and indexes in version 9.6.3 pgaudit Provide detailed session and object audit logging in versions 9.6.3 and 9.5.7 wal2json Output plugin for logical decoding in versions 9.6.3 and 9.5.7 auto_explain Log execution plans of slow statements automatically in versions 9.6.3 and 9.5.7 pg_hint_plan Provides control of execution plans by using hint phrases log_fdw Extension to query your database engine logs within the database pg_freespacemap Examine free space map
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Extension—pgaudit (9.6.3+) • CREATE ROLE rds_pgaudit • Add pgaudit to shared_preload_libraries and pgaudit.role = rds_pgaudit in a custom parameter group in PostgreSQL 9.6 family • Apply the modified parameter group to 9.6.3+ database instance and apply immediately • CREATE EXTENSION pgaudit • Grant SELECT on all tables to rds_pgaudit to enable auditing • GRANT SELECT ON t1 TO rds_pgaudit; • Database logs will show entry as follows • ... 2017-06-12 19:09:49 UTC:…:pgadmin@postgres:[11701]:LOG: AUDIT: OBJECT,1,1,READ,SELECT,TABLE,public.t1,select * from t1; ...
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Extension—pg_stat_statements CREATE EXTENSION pg_stat_statements SELECT * from pg_stat_statements order by total_time DESC; -[ RECORD 2 ]-------+-------- userid | 16388 dbid | 16464 queryid | 4286627671 query | UPDATE pgbench_accounts SET abalance = abalance + ? WHERE aid = ?; calls | 165125 total_time | 5251.54200000001 min_time | 0.015 max_time | 5.558 mean_time | 0.0318034337623008 stddev_time | 0.0369181019548524 rows | 165125 SELECT substring(query, 1, 50) AS short_query, round(total_time::numeric, 2) AS total_time, calls, round(mean_time::numeric, 2) AS mean, round((100 * total_time /sum(total_time::numeric) OVER ())::numeric, 2) AS percentage_cpu FROM pg_stat_statements ORDER BY total_time DESC LIMIT 10;
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Use DB Parameter Groups Create a basic template for all your deployments with rds.force_ssl=true shared_preload_libraries=pgaudit, auto_explain,pg_stat_statements,pg_re pack pgaudit.role = rds_pgaudit huge_pages = on Use basic template or copy to instance- specific parameter group Avoid typos while editing parameters
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Optimizing PostgreSQL Parameters Most instances have a default calculated shared_buffers • Typically 25% of RAM For high active connections require free memory • First make sure huge_pages = on • Reduce shared_bufferpool if required Increase maintenance_work_mem as required Right size work_mem based on typical sort performances • Impact is per connection For high load write transactions • max_wal_size = 1024 (MB)
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cloud-optimized relational database Performance and availability of commercial databases Simplicity and cost effectiveness of open source databases, now with PostgreSQL compatibility What Is Amazon Aurora?
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A S e r v i c e - O r i e n t e d A r c h i t e c t u r e A p p l i e d t o t h e D a t a b a s e Move the logging and storage layer into a multitenant, scale-out, database-optimized storage service. Integrate with other AWS services like Amazon EC2, Amazon VPC, Amazon DynamoDB, Amazon SWF, and Amazon Route 53 for control and monitoring. Make it a managed service—using Amazon RDS. Takes care of management and administrative functions. Amazon DynamoDB Amazon SWF Amazon Route 53 Logging + Storage SQL Transactions Caching Amazon S3 1 2 3 Amazon RDS
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PostgreSQL 9.6 + Amazon Aurora cloud-optimized storage § Performance: 2x–3x higher throughput than PostgreSQL alone § Availability: failover time of <30 seconds § Durability: six copies across three Availability Zones § Read Replicas: single-digit millisecond lag times on up to 15 replicas Amazon Aurora Storage Aurora PostgreSQL
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Aurora Storage Engine Overview Data is replicated six times across three Availability Zones Continuous backup to Amazon S3 (built for 11 9s durability) Continuous monitoring of nodes and disks for repair 10 GB segments as unit of repair or hotspot rebalance Quorum system for read/write; latency tolerant Quorum membership changes do not stall writes Storage volume automatically grows up to 64 TB AZ 1 AZ 2 AZ 3 Amazon S3 Database Node Storage Node Storage Node Storage Node Storage Node Storage Node Storage Node Storage Monitoring
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RDS/Aurora PostgreSQL Monitoring Amazon CloudWatch metrics • CPU utilization • Storage • Memory • Swap usage • DB connections • I/O (read and write) • Latency (read and write) • Throughput (read and write) • Replica lag Amazon CloudWatch alarms • Similar to on-premises monitoring tools Enhanced Monitoring for Amazon RDS • Access to over 50 CPU, memory, file system, and disk I/O metrics • Low as 1 second intervals Integration with third-party monitoring tools
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Performance Insight: Simplifying Monitoring • Dashboard displaying load on database • Easy • Powerful • Identifies source of bottlenecks • Top SQL • Adjustable time frame • Hour, day, week, month • Up to 35 days of data Max CPU
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PostgreSQL Events and Logs
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch Metric—Max Used Trans IDs
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Avoiding Transaction ID Wraparound Two billion “in-flight” un-vacuumed transactions before PostgreSQL takes dramatic action to avoid data loss If the number of un-vacuumed transactions reaches (2^31 – 1,000,000): • PostgreSQL sets the database to read-only mode and requires an offline, single-user, standalone vacuum VACUUM (non blocking) records free space into .fsm files VACUUM FULL (blocking) shrinks the size of the database disk files
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AUTOVACUUM Tuning For 24/7 constant load on database server • AUTOVACUUM may not get a chance to finish its job For high number of tables • Increase autovacuum_max_workers from default 3 to higher number • Increase autovacuum_vacuum_cost_limit • Note: With this change there may be performance impact For large tables use • Decrease autovacuum_vacuum_scale_factor from 0.2 (20%) to 0.05 (5%) • ALTER TABLE myablename SET autovacuum_scale_factor = 0.02
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Extension—pg_repack (9.6.3+) • Add pg_repack to shared_preload_libraries in a custom parameter group in PostgreSQL 9.6 family • Apply the modified parameter group to 9.6.3+ database instance and apply immediately • CREATE EXTENSION pg_repack • Use pg_repack client utility using rds_superuser privileges with –k option • pg_repack -h myproductiondb.cw7jjfgdr4on8.us-west- 2.rds.amazonaws.com -U pgadmin -k postgres
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CloudWatch—Replication Lag
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Read Replica Lag Increase wal_keep_segments to allow replicas to catch up after interruption Use wal_compression = on Use higher checkpoint_timeout Lag for replicas vs long running queries on replicas • max_standby_archive_delay • max_standby_streaming_delay Use hot_standby_feedback = on on Read Replicas Use similar sized instance size (or larger)
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon RDS PostgreSQL Customers
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you!

×