Running your open source database on Amazon EC2 is convenient, but can it be easier, more scalable, and more cost effective? You bet it can. In this session, learn how to effectively and efficiently migrate a PostgreSQL database from Amazon EC2 to Amazon Aurora, increase availability, and reduce cost.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migrating Open Source Databases
from Amazon EC2 to Aurora
PostgreSQL
Joey Kwasniewski
Principal Engineer
ClassPass
D A T 3 3 9
Kevin Jernigan
Principal Product Manager
Amazon

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
About ClassPass
Amazon EC2 PostgreSQL
Migration from Amazon EC2 to Amazon RDS PostgreSQL
Migration from Amazon RDS to Aurora PostgreSQL
Lessons learned

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
About ClassPass
ClassPass is an all-access
membership to a global
network of 10,000 fitness
studios
ClassPass was started in 2013
after the founder struggled to
find a dance class to take after
work in NYC
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ClassPass architecture
Microservices
Over 100 services running many different functions
Most services have stateful data stored in PostgreSQL databases
Many environments
Development and production environments
Team environments
No dedicated database administrators
Teams own management of their own databases
Senior engineers help as issues arise

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EC2 PostgreSQL
Started using a monolithic application
One database isn’t too hard to manage
One small EC2 instance with standard provisioned EBS volume
We start to get more data and more services
Resizing EC2 instances and EBS volumes to keep up with load
Creating new databases to isolate different services from affecting each other
Backups, replicas, and redundancy
Scripts to create EBS snapshots and database dumps
Multiple EC2 instances with manually configured replication configurations

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration from Amazon EC2 to RDS PostgreSQL
Still early in lifecycle of company
Some downtime is okay
Dataset is still not extremely large
At time of migration some AWS services did not yet exist
AWS Database Migration Service (AWS DMS) was not yet released, which would have allowed
us to have less downtime
Aurora PostgreSQL was also not released yet
How we did it
Create new RDS instance
Put the application in maintenance
Take a pg_dump of the database
Load the pg_dump into the new RDS instance

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration from Amazon EC2 to RDS PostgreSQL
> psql -U root -h example.account.us-east1.rds.amazonaws.com --dbname=postgresql
> psql$ create database classpass_example;
> psql$ create role classpass with password ‘example’;
> psql$ grant all on database classpass_example to classpass;
> psql$ q
> pg_dump dbname=classpass_example -f classpass_example.sql
> psql –f classpass_example.sql -h example.account.us-east1.rds.amazonaws.com
–U root –d classpass_example

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration from Amazon EC2 to RDS PostgreSQL
Lots of benefits
Easy to add replicas
Multi-AZ automatic failover
Easier to increase storage capacity
Some scaling pains
Still need to manage storage capacity
IO capacity management can be
complicated
Parameter groups
Database version upgrade management
No base operating system to manage
Some databases getting close to 16 TB

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration from Amazon RDS to Aurora PostgreSQL
Challenges
Large databases with heavy write loads
Mission-critical workflows that need very little downtime
Lots of legacy code using database making architecture changes expensive
Customers facing slowdowns at peak times
What Aurora gives us
Automatic disk capacity growth to 64 TB
Much higher I/O maximums with no need to manually provision
A very easy migration process that allows almost zero downtime

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration from Amazon RDS to Aurora PostgreSQL
How we did it
Create a CNAME in Amazon Route 53 that points to the RDS instance hostname
Update all configurations that accesses the RDS instance to the CNAME
Created a new Aurora parameter group that closely matched RDS parameter group
Turns off nonessential services to limit writes to the RDS instance
Created a Aurora read replica for the RDS master

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration from Amazon RDS to Aurora PostgreSQL
How we did it
Wait for all replication data to be fully synced
Stop all writes to the old RDS instance
Promote the Aurora read replica
Update the CNAME to the Aurora read replica hostname
Stop the old RDS instance
Start all services

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migration from Amazon RDS to Aurora PostgreSQL
0
50
100
150
200
250
300
Workflow 1 Workflow 2 Workflow 3 Workflow 4
Workflow completion time (minutes)
RDS Aurora

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lessons learned
AWS keeps making things easier
If we had to do this again, AWS DMS and Aurora PostgreSQL would have made the initial
migration faster and with less downtime
Aurora doesn’t solve all problems
If you have very large joins you will still see performance issues
Look at optimizing queries or changing work_mem to do operations all in memory
Don’t wait until you are almost out of a resource
We waited too long and had issues with disk capacity on our RDS instance while migrating to
Aurora

20. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Joey Kwasniewski
jkwasniewski@classpass.com

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.