1.
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Amazon Aurora
Relational databases reimagined.
Ronan Guilfoyle, Solutions Architect, AWS
Brian Scanlan, Engineer, Intercom

2.
Current DB Architectures are Monolithic
Multiple layers of
functionality all on a
single box
SQL
Transactions
Caching
Logging

3.
Current DB Architectures are Monolithic
Even when you scale
it out, you’re still
replicating the same
stack
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application

4.
Current DB Architectures are Monolithic
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application Even when you scale
it out, you’re still
replicating the same
stack

5.
Current DB Architectures are Monolithic
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Storage
Application Even when you scale
it out, you’re still
replicating the same
stack

6.
This is a problem.
For cost. For flexibility. And for availability.

7.
Re-imagining the Relational Database
What if we were inventing the database today?
You wouldn’t design it the way we did in 1970. At least not entirely
You’d build something scale-out, self-healing, that leverage existing
AWS services

8.
Relational databases reimagined for the cloud.
speed and availability of high-end commercial databases
simplicity and cost-effectiveness of open source databases
drop-in compatibility with MySQL
simple pay as you go pricing
Delivered as a managed service.

9.
Amazon Aurora
applying a Service-oriented architecture
to the database
• Moved the logging and storage layer
into a multi-tenant, scale-out
database-optimized storage service
• Integrated with other AWS Services
like EC2, VPC, DynamoDB, SWF,
Route 53 for control plane
operations
• Integrated with S3 for continuous
backup and 99.999999999%
durability
Logging + Storage
SQL
Transactions
Caching
Control PlaneData Plane
Amazon S3
DynamoDB
Amazon SWF
Amazon Route 53

10.
Aurora Works with Your Existing Apps

11.
An Established Ecosystem
Business Intelligence Data Integration Query & Monitoring SI & Consulting
“It is great to see Amazon Aurora remains MySQL compatible; we have found our
connectors work with Aurora seamlessly. Today, customers can take our drivers
and connect to Aurora, MariaDB or MySQL without worrying about compatibility. We look
forward to working with the Aurora team in the future to further accelerate innovation within the
MySQL ecosystem.” – Rasmus Johansson, VP Engineering

12.
Amazon Aurora is Easy to Use

13.
Aurora Makes it Easy to Run Your Databases
• Create a database in minutes
• Automatic patching
• Push-button scaling
• Failure detection and failover.
• Read Replica’s are available as failover targets, with no data loss
Amazon
RDS

14.
Aurora simplifies storage management
• Instant creation of user-snapshots
• Continuous backups to S3
• Automatic storage scaling up to 64 TB -
no performance or availability impact
• Automatic restriping, mirror repair, hot
spot management, encryption
Amazon
RDS

15.
Aurora simplifies Data Security
• Encryption to secure data at rest
– AES-256; hardware accelerated
– All blocks on disk and in Amazon S3 encrypted
– Key management via AWS KMS
• SSL to secure data in transit
• Network isolation via Amazon VPC
by default
• No direct access to nodes
• Supports industry standard security
and data protection certifications
AZ 1 AZ 3
Primary
Instance
Amazon S3
Replica
Instance
Customer
VPC
Internal
VPC
MySQL App
AZ 2

16.
Amazon Aurora is Highly Available

17.
Aurora is Highly Available
• Highly available by default
– 6-way replication across 3 AZs
– 4 of 6 write quorum
• Automatic fallback to
3 of 4 if an AZ is unavailable
– 3 of 6 read quorum
• SSD, scale-out, multi-tenant storage
– Seamless storage scalability
– Up to 64TB database size
– Only pay for what you use
• Log-structured storage
– Many small segments, each with
their own redo logs
– Log pages used to generate data pages
– Eliminates chatter between database and storage
SQL
Transaction
s
AZ 1 AZ 2 AZ 3
Caching
Amazon S3

18.
Aurora Performs Consistent, Low Latency Writes
Improvements
• Consistency - tolerance to outliers
• Latency - 2 phase commit vs. asynchronous replication
• Significantly more efficient use of network IO
AZ 1 AZ 2
Primary
Instance
Standby
Instance
EBS
Amazon S3
EBS
mirror
EBS
EBS
mirror
AZ 1 AZ 3
Primary
Instance
Amazon S3
AZ 2
Replica
Instance
Log records
Binlog
Data
Doublewrite buffer
FRM files, metadata
Type of writes
MySQL Multi-AZ with Standby Amazon Aurora
async
4/6 quorum
2 phase commit
PiTR
Sequential
write
Sequential
write Distributed
writes

19.
Aurora Performs Consistent, Low Latency Writes
Improvements
• Consistency - tolerance to outliers
• Latency - 2 phase commit vs. asynchronous replication
• Significantly more efficient use of network IO
AZ 1 AZ 2
Primary
Instance
Standby
Instance
EBS
Amazon S3
EBS
mirror
EBS
EBS
mirror
AZ 1 AZ 3
Primary
Instance
Amazon S3
AZ 2
Replica
Instance
Log records
Binlog
Data
Doublewrite buffer
FRM files, metadata
Type of writes
MySQL with Standby Amazon Aurora
async
4/6 quorum
2 phase commit
PiTR
Sequential
write
Sequential
write Distributed
writes

20.
Self-healing and fault-tolerant
• Lose 2 copies or an AZ failure without read or write availability impact
• Lose 3 copies without read availability impact
• Automatic detection, replication and repair
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
Read & Write AvailabilityRead Availability

21.
Traditional Databases
• Have to replay logs since the
last checkpoint
• Single threaded in MySQL;
requires a large number of
disk accesses
Amazon Aurora
• Underlying storage replays
redo records on demand as
part of a disk read
• Parallel, distributed,
asynchronous
Checkpointed Data Redo Log
Crash at T0 requires
a re-application of the
SQL in the redo log since
last checkpoint
T0 T0
Crash at T0 will result in redo
logs being applied to each segment
on demand, in parallel, asynchronously
Aurora Has Instant Crash Recovery

22.
Aurora’s Cache Survives a DB Restart
• We moved the cache out of
the database process
• Cache remains warm in the
event of a database restart
• Lets you resume fully
loaded operations much
faster
• Instant crash recovery +
survivable cache = quick
and easy recovery from DB
failures
SQL
Transactions
Caching
SQL
Transactions
Caching
SQL
Transactions
Caching
Caching Process is Outside the DB process
and remains warm across a database restart

23.
Multiple failover targets, without data loss.
MySQL Read Scaling
• Replicas must replay logs
• Replicas place additional load on master
• Replica lag can grow indefinitely
• Failover results in data loss
Page cache
invalidation
Aurora Master
30% Read
70% Write
Aurora Replica
100% New
Reads
Shared Multi-AZ Storage
MySQL Master
30% Read
70% Write
MySQL Replica
30% New Reads
70% Write
Single
threaded
binlog apply
Data Volume Data Volume

24.
You Can Simulate Failures Using SQL
• To cause the failure of a component at the database node:
ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}]
• To simulate the failure of disks:
ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN
[DISK index | NODE index] FOR INTERVAL interval
• To simulate the failure of networking:
ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type
[TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval

25.
Intermission…

26.
Intercom
brian.scanlan@intercom.io

27.
Intercom’s technology
- Largely monolithic Ruby on Rails application. Strong culture
around Continuous Deployment and general DevOps best
practices. Some adoption of SOA/Microservices.
- Built using MySQL and Ruby on Rails’ ActiveRecord ORM
for rapid development. Unstructured customer data stored
in MongoDB, however all messages in MySQL.
- Heavy use of AWS services and other SaaS services (New
Relic, Code Climate, CodeShip, LogEntries).
- Custom infrastructure/code orchestration & deployment
system.

28.
Recent MySQL woes
- Highly sensitive to MySQL performance and started experiencing regular inexplicable
performance degradation. Could not vertically scale our way out of the problem. Engaged
RDS Support & MySQL consultants.
- Adjusted parameters e.g. lock wait timeouts, transaction read isolation levels and
txn_flush_at_commit, etc. Instrumented and reduced number of long-running transactions in
an attempt to reduce lock contention.
- Greatly increased the number of MySQL metrics being collected, built application level
fingerprinting and automated data collection during outages. Got our hands dirty with
MySQL’s performance schema. Reducing read throughput stabilised.

29.
Why we’re interested in Aurora
- While we bought ourselves time with improved caching, Aurora
gives us more options and a lot more vertical scaling opportunities.
- Operational experiences of using read-replicas with RDS/MySQL
means we don’t trust them for customer facing queries. The
eventual consistency guarantees of Aurora look good enough for our
application to use for practically all read queries.

30.
intercom.io
blog.intercom.io
team@intercom.io
brian.scanlan@intercom.io

31.
Enterprise grade features and performance
at open source prices

32.
Aurora Pricing
Simple pricing
• No licenses
• No lock-in
• Pay only for what you use
Discounts
• 44% with a 1 year RI
• 63% with a 3 year RI
vCPU Mem Hourly Price
db.r3.large 2 15.25 $0.29
db.r3.xlarge 4 30.5 $0.58
db.r3.2xlarge 8 61 $1.16
db.r3.4xlarge 16 122 $2.32
db.r3.8xlarge 32 244 $4.64
• Storage consumed, up to 64TB, is $0.10/GB/month
• IOs consumed are billed at $0.20 per million IO
• Prices are for Virginia

33.
Aurora – Enterprise Grade. Open Source Prices
• Expanding to unlimited preview
• Adding preview support for US West (Oregon) and EU (Ireland)
• Signup for preview access at:
https://aws.amazon.com/rds/aurora/preview
• Full service launch in the coming months

34.
LONDON