Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Migrating Postgres from EC2 to RDS

3,416 views

Published on

How to seamlessly migrate your EC2 Postgres to RDS using Bucardo's MultiMaster replication.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Migrating Postgres from EC2 to RDS

  1. 1. Migrating to RDS Seamless Migrations using Bucardo
  2. 2. –Everyone Why the F@#$%@#$% would I want to use RDS?
  3. 3. • Difficult to scale • Difficult to recover • Difficult to failover • Expensive • Complicated
  4. 4. –Jack Burton Sit tight, hold the fort and keep the home fires burning. And if we're not back by dawn... call the president.
  5. 5. –Jack Burton Are you crazy... Is that your problem?
  6. 6. Create your RDS Instance Use RDSTune to get a default parameter set https://bitbucket.org/davidkerr/rdstune > gem install rdstune > rdstune -t web -m 2.75 -c 200
  7. 7. Create your RDS Instance
  8. 8. Create your RDS Instance
  9. 9. Other best practices 1 Database per RDS Instance Use Multi-AZ Deployments for production Use SSDs/Provisioned IOPS
  10. 10. Create Users One user will be your replication user (usually the app user / database owner) Bucardo setup says that this user needs access to session_replication_role parameter which is granted by the rds_superuser role.
  11. 11. Session Replication Role The session_replication_role parameter allows bucardo to disable triggers. Which is critical for Bucardo to manage replication.
  12. 12. Be Aware export PGOPTIONS='-c session_replication_role=replica' psql -h mydb.mycompany.com -U dba mydb Password for user dba: psql: FATAL: permission denied to set parameter "session_replication_role" RDS is finicky in setting the session_replica_role mydb=> set session_replication_role=replica; SET mydb=> show session_replication_role; session_replication_role -------------------------- replica (1 row)
  13. 13. Prep your RDS Instance > pg_dump -s -n <schema> <database> | psql -h <rds host> -U <rds user> <database> • Set the database owner • Create any extensions • Create an empty schema
  14. 14. Bucardo Bucardo provides is Multi-Master replication for Postgres This is logical replication (trigger based) and won’t interfere with Postgres streaming replication Is Asynchronous (which is perfect for this) You can find installation instructions here https://bucardo.org/wiki/Bucardo#Obtaining_Bucardo
  15. 15. Configuration > createdb bucardo > bucardo install > bucardo add db local dbname=myDb addalltables > bucardo add db remote db=myDb pghost=rds.xxyzrtl2.us-west-2.rds.amazonaws.com > bucardo add dbgroup RDS local:source remote:source > bucardo add sync RDSSync tables=all dbs=RDS autokick=0
  16. 16. Copy the Data > pg_dump -Fc --data-only -N bucardo myDb | pg_restore -Fc --disable-triggers -U myUser -h rds.xxyzrtl2.us-west-2.rds.amazonaws.com -d myDB
  17. 17. Fix your sequences CREATE or REPLACE FUNCTION bsequencer(sequence_name text,offset_by int,even boolean) RETURNS VOID AS $$ DECLARE myval bigint; BEGIN EXECUTE 'select last_value +'||offset_by|| ' from '||sequence_name INTO myval; IF ( even AND ( (myval % 2) != 0) ) OR ( NOT even AND ( (myval % 2) = 0) ) THEN myval := myval + 1; END IF; EXECUTE 'SELECT setval( ''' || sequence_name || ''',' || myval || ')'; EXECUTE 'ALTER sequence ' || sequence_name || ' increment by 2'; END; $$ LANGUAGE plpgsql; local => select bsequencer('sequence_name',100,true); RDS => select bsequencer('sequence_name',100,false);
  18. 18. /Kick your Bucardo > bucardo update sync RDSSync autokick=1 > bucardo reload config
  19. 19. Repoint your Apps • Repoint the DNS entry for your database host to the RDS host. • Apps will slowly switch over to the RDS host as connections age and get re-established • Or implement close to a release to force a reconnect
  20. 20. Tear down the old • Once all of the connections have drained off the old DBs you can de-commission them. • Remember to remove Bucardo!
  21. 21. Thanks!

×