If you’re running a MySQL database at scale, there’s a good chance you’re sharding your database deployment. Sharding is a useful way to increase the scale of your deployment, but it has drawbacks like higher costs, high administration overheard and lower elasticity. It’s harder to grow or shrink a sharded database deployment to match your traffic patterns. In this session, we will discuss and demonstrate how to use AWS Database Migration Service to consolidate multiple MySQL shards into an Amazon Aurora cluster to reduce cost, improve elasticity and make it easier to manage your database. Learning Objectives: Learn how to scale your MySQL database at reduced cost and higher elasticity, by consolidating multiple shards into one Amazon Aurora cluster.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
February 21, 2017
Consolidating MySQL Shards in Aurora
AWS Database Migration Service

2. Agenda
• How does the cloud help?
• Problem statement
• Amazon Aurora overview
• Introduction to AWS DMS
• Proposed problem solution
• Show me!

3. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cloud benefits

4. • Multi-engine support: Aurora, MySQL, MariaDB,
PostgreSQL, Oracle, SQL Server
• Automated provisioning, patching, scaling,
backup/restore, failover
• High availability with RDS Multi-AZ
– 99.95% SLA for Multi-AZ deployments
Amazon RDS

5. • Lower TCO because we manage the muck
• Get more leverage from your teams
• Focus on the things that differentiate you
• Built-in high availability and cross region replication
across multiple data centers
• Available on all engines, including base/standard
editions, not just for enterprise editions
• Now even a small startup can leverage multiple data
centers to design highly available apps with over
99.95% availability.
The Cloud makes things cheaper, easier, better

6. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How did we end up here?

7. Things change
In the beginning…
- The system ran fine but then growth happened
To solve the problem you could:
- Scale up
- Scale out

8. Sharding

9. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Aurora

10. Relational databases were not designed for the cloud
Multiple layers of
functionality all in a
monolithic stack
SQL
Transactions
Caching
Logging

11. Not much has changed in last 20 years
Even when you scale it out, you’re still replicating the same stack
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Storage
Application

12. Scale-out, distributed, log structured storage
Master Replica Replica Replica
Availability Zone 1
Shared Storage Volume – Transaction Aware
Primary
Database
Node
Read
Replica /
Secondary
Node
Read
Replica /
Secondary
Node
Read
Replica /
Secondary
Node
Availability Zone 2 Availability Zone 3
AWS Region
Storage
Monitoring
Database and
Instance
Monitoring

13. Meet Amazon Aurora ……
Database reimagined for the cloud
 Speed and availability of high-end commercial databases
 Simplicity and cost-effectiveness of open source databases
 Drop-in compatibility with MySQL
 Simple pay as you go pricing
Delivered as a managed service

14. Enterprise-class performance
• Provides 5X the throughput
of standard MySQL running
on the same hardware.
• Achieve up to 585,000
reads and 100,000 writes
per second
• Read replicas with <10ms
latency

15. Aurora Scaling
With user connection With number of tables
With database size - SYSBENCH With database size - TPCC
Connections
Amazon
Aurora
RDS MySQL
w/ 30K IOPS
50 40,000 10,000
500 71,000 21,000
5,000 110,000 13,000
Tables
Amazon
Aurora
MySQL
I2.8XL
local SSD
RDS MySQL
w/ 30K IOPS
(single AZ)
10 60,000 18,000 25,000
100 66,000 19,000 23,000
1,000 64,000 7,000 8,000
10,000 54,000 4,000 5,000
8x
U P T O
F A S T E R
11x
U P T O
F A S T E R
DB Size
Amazon
Aurora
RDS MySQL
w/ 30K IOPS
1GB 107,000 8,400
10GB 107,000 2,400
100GB 101,000 1,500
1TB 26,000 1,200
DB Size Amazon Aurora
RDS MySQL
w/ 30K IOPS
80GB 12,582 585
800GB 9,406 69
21
U P T O
F A S T E R
136x
U P T O
F A S T E R

16. Real-life data – gaming workload
Aurora vs. RDS MySQL – r3.4XL, MAZ
Aurora 3X faster on r3.4xlarge

17. Cost of ownership: Aurora vs. MySQL
MySQL configuration hourly cost
Primary
r3.8XL
Standby
r3.8XL
Replica
r3.8XL
Replica
R3.8XL
Storage
6 TB / 10 K PIOP
Storage
6 TB / 10 K PIOP
Storage
6 TB / 5 K PIOP
Storage
6 TB / 5 K PIOP
$1.33/hr
$1.33/hr
$1.33/hr $1.33/hr
$2.42/hr
$2.42/hr $2.42/hr
Instance cost: $5.32 / hr
Storage cost: $8.30 / hr
Total cost: $13.62 / hr
$2,42/hr

18. Cost of ownership: Aurora vs. MySQL
Aurora configuration hourly cost
Instance cost: $4.86 / hr
Storage cost: $4.43 / hr
Total cost: $9.29 / hr
Primary
r3.8XL
Replica
r3.8XL
Replica
R3.8XL
Storage / 6 TB
$1.62 / hr $1.62 / hr $1.62 / hr
$4.43 / hr
*At a macro level Aurora saves over 50% in
storage cost compared to RDS MySQL.
31.8%
Savings
 No idle standby instance
 Single shared storage volume
 No PIOPs – pay for use I/O
 Reduction in overall IOP

19. Cost of ownership: Aurora vs. MySQL
Further opportunity for saving
Instance cost: $2.43 / hr
Storage cost: $4.43 / hr
Total cost: $6.86 / hrStorage IOPs assumptions:
1. Average IOPs is 50% of Max IOPs
2. 50% savings from shipping logs vs. full pages
49.6%
Savings
Primary
r3.8XL
Replica
r3.8XL
Replica
r3.8XL
Storage / 6TB
$0.81 / hr $0.81 / hr $0.81 / hr
$4.43 / hr
r3.4XL r3.4XL r3.4XL
 Use smaller instance size
 Pay-as-you-go storage

20. Use case: MySQL shard consolidation
Master
Read
Replica
Shared distributed
storage volume
M S
M M
M
S S
S
MySQL shards Aurora cluster
Customer, a global SAAS provider, was using hundreds of MySQL shards in order to avoid MySQL
performance and connection scalability bottlenecks
 Consolidated multiple 29 MySQL shards to single r3.4xlarge Aurora cluster
 Even after consolidation cluster utilization is still 30% with plenty of headroom to grow.

21. Amazon Aurora Customers

22. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Database Migration Service

23. AWS Database Migration Service (AWS DMS)
DMS migrates databases to AWS easily and securely
with minimal downtime. It can migrate your data to and
from most widely used commercial and open-source
databases – and for as little a $3 for TB DB.
Amazon Aurora

24. Customer
premises
Application users
AWS
Internet
VPN
Start a replication instance
Connect to source and target
databases
Select tables, schemas, or
databases
 Let AWS DMS create tables,
load data, and keep them in
sync
 Switch applications over to
the target at your convenience
Keep your apps running during the migration
AWS
DMS

25. Load is table by table
Replication instance
Source Target

26. Change data capture (CDC) and apply
Replication instance
Source Target
Update
t1 t2
t1
t2
Transactions Change
apply
after bulk
load

27. Replication
instance
Source Target
What else can I do?
Source
Source

28. Take it all—or not
Source Target
Replication instance
instance

29. AWS Database Migration Service Customers

30. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can we make it better?

31. Establish a beach head
Before After

32. Validate

33. Partial Migration

34. The Result

35. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo

36. Thank you!
aws.amazon.com/dms
aws.amazon.com/rds/aurora
Learn more..