Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved
Amazon Aurora
Relational databases reimagi...
Current DB Architectures are Monolithic
Multiple layers of
functionality all on a
single box
SQL
Transactions
Caching
Logg...
Current DB Architectures are Monolithic
Even when you scale
it out, you’re still
replicating the same
stack
SQL
Transactio...
Current DB Architectures are Monolithic
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Application Even...
Current DB Architectures are Monolithic
SQL
Transactions
Caching
Logging
SQL
Transactions
Caching
Logging
Storage
Applicat...
This is a problem.
For cost. For flexibility. And for availability.
Re-imagining the Relational Database
What if we were inventing the database today?
You wouldn’t design it the way we did i...
Relational databases reimagined for the cloud.
speed and availability of high-end commercial databases
simplicity and cost...
Amazon Aurora
applying a Service-oriented architecture
to the database
•  Moved the logging and storage layer
into a multi...
Aurora Works with Your Existing Apps
An Established Ecosystem
Business Intelligence Data Integration Query & Monitoring SI & Consulting
“It is great to see Ama...
Amazon Aurora is Easy to Use
Aurora Makes it Easy to Run Your Databases
•  Create a database in minutes
•  Automatic patching
•  Push-button scaling
• ...
Aurora simplifies storage management
•  Instant creation of user-snapshots
•  Continuous backups to S3
•  Automatic storag...
Aurora simplifies Data Security
•  Encryption to secure data at rest
–  AES-256; hardware accelerated
–  All blocks on dis...
Amazon Aurora is Highly Available
Aurora is Highly Available
•  Highly available by default
–  6-way replication across 3 AZs
–  4 of 6 write quorum
•  Auto...
Aurora Performs Consistent, Low Latency Writes
Improvements
•  Consistency - tolerance to outliers
•  Latency - 2 phase co...
Aurora Performs Consistent, Low Latency Writes
Improvements
•  Consistency - tolerance to outliers
•  Latency - 2 phase co...
Self-healing and fault-tolerant
•  Lose 2 copies or an AZ failure without read or write availability impact
•  Lose 3 copi...
Traditional Databases
•  Have to replay logs since the
last checkpoint
•  Single threaded in MySQL;
requires a large numbe...
Aurora’s Cache Survives a DB Restart
•  We moved the cache out of
the database process
•  Cache remains warm in the
event ...
Multiple failover targets, without data loss.
MySQL Read Scaling
•  Replicas must replay logs
•  Replicas place additional...
You Can Simulate Failures Using SQL
•  To cause the failure of a component at the database node:
ALTER SYSTEM CRASH [{INST...
Intermission…	
  
Intercom
brian.scanlan@intercom.io
Intercom’s technology
-  Largely monolithic Ruby on Rails application. Strong culture
around Continuous Deployment and gen...
Recent MySQL woes
-  Highly sensitive to MySQL performance and started experiencing regular inexplicable
performance degra...
Why we’re interested in Aurora
-  While we bought ourselves time with improved caching, Aurora
gives us more options and a...
intercom.io
blog.intercom.io
team@intercom.io
brian.scanlan@intercom.io
Enterprise grade features and performance
at open source prices
Aurora Pricing
Simple pricing
•  No licenses
•  No lock-in
•  Pay only for what you use
Discounts
•  44% with a 1 year RI
...
Aurora – Enterprise Grade. Open Source Prices
•  Expanding to unlimited preview
•  Adding preview support for US West (Ore...
LONDON
Amazon Aurora
Upcoming SlideShare
Loading in …5
×

Amazon Aurora

7,305 views

Published on

Amazon Aurora is a MySQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. The service is now in preview. Come to our session for an overview of the service and learn how Aurora delivers up to five times the performance of MySQL yet is priced at a fraction of what you'd pay for a commercial database with similar performance and availability.

Speakers:
Ronan Guilfoyle, AWS Solutions Architect
Brian Scanlan, Engineer, Intercom.io

Published in: Technology
  • Be the first to comment

Amazon Aurora

  1. 1. ©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved Amazon Aurora Relational databases reimagined. Ronan Guilfoyle, Solutions Architect, AWS Brian Scanlan, Engineer, Intercom
  2. 2. Current DB Architectures are Monolithic Multiple layers of functionality all on a single box SQL Transactions Caching Logging
  3. 3. Current DB Architectures are Monolithic Even when you scale it out, you’re still replicating the same stack SQL Transactions Caching Logging SQL Transactions Caching Logging Application
  4. 4. Current DB Architectures are Monolithic SQL Transactions Caching Logging SQL Transactions Caching Logging Application Even when you scale it out, you’re still replicating the same stack
  5. 5. Current DB Architectures are Monolithic SQL Transactions Caching Logging SQL Transactions Caching Logging Storage Application Even when you scale it out, you’re still replicating the same stack
  6. 6. This is a problem. For cost. For flexibility. And for availability.
  7. 7. Re-imagining the Relational Database What if we were inventing the database today? You wouldn’t design it the way we did in 1970. At least not entirely You’d build something scale-out, self-healing, that leverage existing AWS services
  8. 8. Relational databases reimagined for the cloud. speed and availability of high-end commercial databases simplicity and cost-effectiveness of open source databases   drop-in compatibility with MySQL   simple pay as you go pricing Delivered as a managed service.
  9. 9. Amazon Aurora applying a Service-oriented architecture to the database •  Moved the logging and storage layer into a multi-tenant, scale-out database-optimized storage service •  Integrated with other AWS Services like EC2, VPC, DynamoDB, SWF, Route 53 for control plane operations •  Integrated with S3 for continuous backup and 99.999999999% durability Logging + Storage SQL Transactions Caching Control PlaneData Plane Amazon S3 DynamoDB Amazon SWF Amazon Route 53
  10. 10. Aurora Works with Your Existing Apps
  11. 11. An Established Ecosystem Business Intelligence Data Integration Query & Monitoring SI & Consulting “It is great to see Amazon Aurora remains MySQL compatible; we have found our connectors work with Aurora seamlessly. Today, customers can take our drivers and connect to Aurora, MariaDB or MySQL without worrying about compatibility. We look forward to working with the Aurora team in the future to further accelerate innovation within the MySQL ecosystem.” – Rasmus Johansson, VP Engineering
  12. 12. Amazon Aurora is Easy to Use
  13. 13. Aurora Makes it Easy to Run Your Databases •  Create a database in minutes •  Automatic patching •  Push-button scaling •  Failure detection and failover. •  Read Replica’s are available as failover targets, with no data loss Amazon RDS
  14. 14. Aurora simplifies storage management •  Instant creation of user-snapshots •  Continuous backups to S3 •  Automatic storage scaling up to 64 TB - no performance or availability impact •  Automatic restriping, mirror repair, hot spot management, encryption Amazon RDS
  15. 15. Aurora simplifies Data Security •  Encryption to secure data at rest –  AES-256; hardware accelerated –  All blocks on disk and in Amazon S3 encrypted –  Key management via AWS KMS •  SSL to secure data in transit •  Network isolation via Amazon VPC by default •  No direct access to nodes •  Supports industry standard security and data protection certifications AZ 1 AZ 3 Primary Instance Amazon S3 Replica Instance Customer VPC Internal VPC MySQL App AZ 2
  16. 16. Amazon Aurora is Highly Available
  17. 17. Aurora is Highly Available •  Highly available by default –  6-way replication across 3 AZs –  4 of 6 write quorum •  Automatic fallback to 3 of 4 if an AZ is unavailable –  3 of 6 read quorum •  SSD, scale-out, multi-tenant storage –  Seamless storage scalability –  Up to 64TB database size –  Only pay for what you use •  Log-structured storage –  Many small segments, each with their own redo logs –  Log pages used to generate data pages –  Eliminates chatter between database and storage SQL Transaction s AZ 1 AZ 2 AZ 3 Caching Amazon S3
  18. 18. Aurora Performs Consistent, Low Latency Writes Improvements •  Consistency - tolerance to outliers •  Latency - 2 phase commit vs. asynchronous replication •  Significantly more efficient use of network IO AZ 1 AZ 2 Primary Instance Standby Instance EBS Amazon S3 EBS mirror EBS EBS mirror AZ 1 AZ 3 Primary Instance Amazon S3 AZ 2 Replica Instance Log records Binlog Data Doublewrite buffer FRM files, metadata Type of writes MySQL Multi-AZ with Standby Amazon Aurora async 4/6 quorum 2 phase commit PiTR Sequential write Sequential write Distributed writes
  19. 19. Aurora Performs Consistent, Low Latency Writes Improvements •  Consistency - tolerance to outliers •  Latency - 2 phase commit vs. asynchronous replication •  Significantly more efficient use of network IO AZ 1 AZ 2 Primary Instance Standby Instance EBS Amazon S3 EBS mirror EBS EBS mirror AZ 1 AZ 3 Primary Instance Amazon S3 AZ 2 Replica Instance Log records Binlog Data Doublewrite buffer FRM files, metadata Type of writes MySQL with Standby Amazon Aurora async 4/6 quorum 2 phase commit PiTR Sequential write Sequential write Distributed writes
  20. 20. Self-healing and fault-tolerant •  Lose 2 copies or an AZ failure without read or write availability impact •  Lose 3 copies without read availability impact •  Automatic detection, replication and repair SQL Transaction AZ 1 AZ 2 AZ 3 Caching SQL Transaction AZ 1 AZ 2 AZ 3 Caching Read & Write AvailabilityRead Availability
  21. 21. Traditional Databases •  Have to replay logs since the last checkpoint •  Single threaded in MySQL; requires a large number of disk accesses Amazon Aurora •  Underlying storage replays redo records on demand as part of a disk read •  Parallel, distributed, asynchronous Checkpointed Data Redo Log Crash at T0 requires a re-application of the SQL in the redo log since last checkpoint T0 T0 Crash at T0 will result in redo logs being applied to each segment on demand, in parallel, asynchronously Aurora Has Instant Crash Recovery
  22. 22. Aurora’s Cache Survives a DB Restart •  We moved the cache out of the database process •  Cache remains warm in the event of a database restart •  Lets you resume fully loaded operations much faster •  Instant crash recovery + survivable cache = quick and easy recovery from DB failures SQL Transactions Caching SQL Transactions Caching SQL Transactions Caching Caching Process is Outside the DB process and remains warm across a database restart
  23. 23. Multiple failover targets, without data loss. MySQL Read Scaling •  Replicas must replay logs •  Replicas place additional load on master •  Replica lag can grow indefinitely •  Failover results in data loss Page cache invalidation Aurora Master 30% Read 70% Write Aurora Replica 100% New Reads Shared Multi-AZ Storage MySQL Master 30% Read 70% Write MySQL Replica 30% New Reads 70% Write Single threaded binlog apply Data Volume Data Volume
  24. 24. You Can Simulate Failures Using SQL •  To cause the failure of a component at the database node: ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}] •  To simulate the failure of disks: ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN [DISK index | NODE index] FOR INTERVAL interval •  To simulate the failure of networking: ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type [TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval
  25. 25. Intermission…  
  26. 26. Intercom brian.scanlan@intercom.io
  27. 27. Intercom’s technology -  Largely monolithic Ruby on Rails application. Strong culture around Continuous Deployment and general DevOps best practices. Some adoption of SOA/Microservices. -  Built using MySQL and Ruby on Rails’ ActiveRecord ORM for rapid development. Unstructured customer data stored in MongoDB, however all messages in MySQL. -  Heavy use of AWS services and other SaaS services (New Relic, Code Climate, CodeShip, LogEntries). -  Custom infrastructure/code orchestration & deployment system.
  28. 28. Recent MySQL woes -  Highly sensitive to MySQL performance and started experiencing regular inexplicable performance degradation. Could not vertically scale our way out of the problem. Engaged RDS Support & MySQL consultants. -  Adjusted parameters e.g. lock wait timeouts, transaction read isolation levels and txn_flush_at_commit, etc. Instrumented and reduced number of long-running transactions in an attempt to reduce lock contention. -  Greatly increased the number of MySQL metrics being collected, built application level fingerprinting and automated data collection during outages. Got our hands dirty with MySQL’s performance schema. Reducing read throughput stabilised.
  29. 29. Why we’re interested in Aurora -  While we bought ourselves time with improved caching, Aurora gives us more options and a lot more vertical scaling opportunities. -  Operational experiences of using read-replicas with RDS/MySQL means we don’t trust them for customer facing queries. The eventual consistency guarantees of Aurora look good enough for our application to use for practically all read queries.
  30. 30. intercom.io blog.intercom.io team@intercom.io brian.scanlan@intercom.io
  31. 31. Enterprise grade features and performance at open source prices
  32. 32. Aurora Pricing Simple pricing •  No licenses •  No lock-in •  Pay only for what you use Discounts •  44% with a 1 year RI •  63% with a 3 year RI vCPU Mem Hourly Price db.r3.large 2 15.25 $0.29 db.r3.xlarge 4 30.5 $0.58 db.r3.2xlarge 8 61 $1.16 db.r3.4xlarge 16 122 $2.32 db.r3.8xlarge 32 244 $4.64 •  Storage consumed, up to 64TB, is $0.10/GB/month •  IOs consumed are billed at $0.20 per million IO •  Prices are for Virginia
  33. 33. Aurora – Enterprise Grade. Open Source Prices •  Expanding to unlimited preview •  Adding preview support for US West (Oregon) and EU (Ireland) •  Signup for preview access at: https://aws.amazon.com/rds/aurora/preview •  Full service launch in the coming months
  34. 34. LONDON

×