Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Terraforming RDS

3,302 views

Published on

How Instacart manages a large fleet of RDS instances with Terraform

Published in: Engineering
  • Be the first to comment

Terraforming RDS

  1. 1. Terraforming RDS
  2. 2. ● What are RDS and Terraform? ● Why does managing RDS with Terraform suck? ● Why do I highly recommend managing RDS with Terraform?
  3. 3. What is RDS?
  4. 4. What is RDS?
  5. 5. What is Terraform?
  6. 6. What is Terraform?
  7. 7. Abstraction
  8. 8. Abstraction
  9. 9. Terraform and RDS defaults resource "aws_db_instance" "muffy-test" { allocated_storage = 100 db_subnet_group_name = "db-subnetgrp" engine = "postgres" engine_version = "11.5" identifier = "muffy-test" instance_class = "db.m5.large" password = "password" skip_final_snapshot = true storage_encrypted = true username = "postgres" } # aws_db_instance.muffy-test will be created + resource "aws_db_instance" "muffy-test" { + address = (known after apply) + allocated_storage = 100 + apply_immediately = (known after apply) . . . + skip_final_snapshot = true + status = (known after apply) + storage_encrypted = true storage_type = "gp2" username = "postgres" vpc_security_group_ids = [ "sg-81f064e5", ] }
  10. 10. Terraform and RDS defaults # aws_db_instance.muffy-test: resource "aws_db_instance" "muffy-test" { address = "muffy-test.....com" allocated_storage = 100 arn = "arn:...:muffy-test" auto_minor_version_upgrade = true availability_zone = "us-east-1b" backup_retention_period = 0 backup_window = "04:40-05:10" ca_cert_identifier = "rds-ca-2019" copy_tags_to_snapshot = false db_subnet_group_name = "db-subnetgrp" delete_automated_backups = true deletion_protection = false endpoint = "muffy-test...:5432" engine = "postgres" engine_version = "11.5" hosted_zone_id = "Z2R2ITUGPM61AM" iam_database_authentication_enabled = false id = "muffy-test" identifier = "muffy-test" instance_class = "db.m5.large" iops = 0 kms_key_id = "arn:...:key/..." license_model = "postgresql-license" maintenance_window = "sat:08:12-sat:08:42" max_allocated_storage = 0 monitoring_interval = 0 multi_az = false option_group_name = "default:postgres-11" parameter_group_name = "default.postgres11" password = (sensitive value) performance_insights_enabled = false performance_insights_retention_period = 0 port = 5432 publicly_accessible = false replicas = [] resource_id = "db-EJHF7...W6VLWRE" skip_final_snapshot = true status = "available" storage_encrypted = true storage_type = "gp2" username = "postgres" vpc_security_group_ids = [ "sg-81f064e5", ] }
  11. 11. Applying changes resource "aws_db_instance" "muffy-test" { allocated_storage = 16000 apply_immediately = false [...] } # aws_db_instance.muffy-test will be updated in-place ~ resource "aws_db_instance" "muffy-test" { ~ allocated_storage = 100 -> 16000 + apply_immediately = false } . . . aws_db_instance.muffy-test: Modifying... [id=muffy-test] ...muffy-test: Modifications complete after 32s [id=muffy-test] Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
  12. 12. Parameter groups # aws_db_parameter_group.muffy-pg must be replaced -/+ resource "aws_db_parameter_group" "muffy-pg" { description = "Managed by Terraform" ~ family = "postgres9.6" -> "postgres11" # forces replacement } aws_db_parameter_group.muffy-pg: Destroying... [id=terraform-20200115031710299600000001] . . . [id=terraform-20200115031710299600000001, 2m50s elapsed] Error: Error deleting DB parameter group: InvalidDBParameterGroupState: One or more database instances are still members of this parameter group terraform-20200115031710299600000001, so the group cannot be deleted status code: 400, request id: 0e99a7be-4b2d-43d7-ac96-5b18af81c307
  13. 13. Parameter apply_method resource "aws_db_parameter_group" "muffy-pg" { family = "postgres11" parameter { apply_method = "immediate" name = "autovacuum_naptime" value = "30" } parameter { apply_method = "pending-reboot" name = "autovacuum_max_workers" value = "15" } } # aws_db_parameter_group.muffy-pg will be updated in-place ~ resource "aws_db_parameter_group" "muffy-pg" { + parameter { + apply_method = "immediate" + name = "autovacuum_naptime" + value = "30" } - parameter { - apply_method = "immediate" -> null - name = "autovacuum_naptime" -> null - value = "15" -> null } }
  14. 14. Plans aside # aws_db_parameter_group.muffy-pg will be updated in-place ~ resource "aws_db_parameter_group" "muffy-pg" { [...] + parameter { + apply_method = "immediate" + name = "autovacuum_naptime" + value = "30" } - parameter { - apply_method = "immediate" -> null - name = "autovacuum_naptime" -> null - value = "15" -> null } parameter { apply_method = "pending-reboot" name = "autovacuum_max_workers" value = "15" } }
  15. 15. A real pain
  16. 16. What’s the real value? # PostgreSQL 9.6 postgres=> select name, setting, unit from pg_settings where name = 'max_wal_size'; name | setting | unit --------------+---------+------ max_wal_size | 128 | 16MB (1 row) postgres=> show max_wal_size; max_wal_size -------------- 2GB (1 row) # PostgreSQL 10 postgres=> select name, setting, unit from pg_settings where name = 'max_wal_size'; name | setting | unit --------------+---------+------ max_wal_size | 128 | MB (1 row) postgres=> show max_wal_size; max_wal_size -------------- 12MB (1 row)
  17. 17. Modules: Good or evil?
  18. 18. Terraform and RDS: Do it!
  19. 19. Muffy Barkocy muffybarkocy@gmail.com twitter.com/muffyb

×