Scale Up and Modernize Your Database with Amazon Relational Database Service and Aurora

P U B L I C S E C T O R
S U M M I T
WASHINGTON, DC

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Scale up and Modernize with Amazon
Aurora and Amazon Relational
Database Service (Amazon RDS)
3 2 0 7 0 2
Kwesi Edwards
Principal Business Development Manager
AWS

S U M M I T
Agenda
Introduction to Amazon RDS and Aurora
Performance and scalability
Availability and reliability
Manageability
Security
How to Migrate

S U M M I T
Amazon Relational Database Service (Amazon RDS)
Choice Value Innovation

S U M M I T
Shared responsibility options
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
App optimization
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
App optimization
Power, HVAC, net
Rack & stack
Server maintenance
OS patches
DB s/w patches
Database backups
Scaling
High availability
DB s/w installs
OS installation
App optimization
On premise EC2 RDS

S U M M I T
Amazon RDS
Choice of open source and commercial databases
Amazon RDS Platform
Open Source Engines Commercial Engines
Advanced monitoring
Routine maintenance
Push-button scaling
Automatic fail-over
Backup & recovery
X-region replication
Isolation & security
Industry compliance
Automated patching
Cloud Native Engine

S U M M I T
Amazon RDS New Features
• Amazon RDS now supports per-second billing
• Amazon RDS for Oracle, MySQL, MariaDB, and PostgreSQL increase
maximum storage size to 64 TiB and I/O performance to 80,000 IOPS
• Amazon RDS for SQL Server Increases the Database Limit Per Database
Instance up to 100
• Amazon RDS for SQL Server Now Supports Always On Availability
Groups

S U M M I T
Amazon Aurora
Amazon Aurora
Speed and availability of high-end commercial databases
Simplicity and cost-effectiveness of open source databases
Drop-in compatibility with MySQL and PostgreSQL
Simple pay as you go pricing
Enterprise database at open source price,
delivered as a managed service

S U M M I T
Amazon Aurora innovations
Re-imagining databases for the cloud
Manageability and security
Performance and scalability
Availability and reliability

S U M M I T
5x faster than MySQL; 3x faster than PostgreSQL

S U M M I T
0
50,000
100,000
150,000
200,000
250,000
MySQL 5.6 MySQL 5.7 MySQL 8.0 Aurora 5.6 Aurora 5.7
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
MySQL 5.6 MySQL 5.7 MySQL 8.0 Aurora 5.6 Aurora 5.7
Write throughput Read throughput
Using Sysbench with 250 tables and 200,000 rows per table on R4.16XL
Write and read throughput
Aurora MySQL is 5x faster than MySQL

S U M M I T
Bulk data load performance
Aurora PostgreSQL loads data 2x faster than PostgreSQL
Copy In
Copy In
Vacuum
Vacuum
Index Build
Index Build
0 500 1000 1500 2000 2500 3000 3500 4000
PostgreSQL
Amazon Aurora
Runtime (seconds)
pgbench initialization, scale 10000 (150 GiB)
86% reduction in vacuum time

S U M M I T
How did we achieve this?
Do less work:
Do fewer I/Os
Minimize network packets
Cache prior results
Offload the database engine
Be more efficient:
Process asynchronously
Reduce latency path
Use lock-free data structures
Batch operations together
Databases are all about I/O
Network-attached storage is all about packets/second
High-throughput processing is all about context switches

S U M M I T
Aurora parallel query processing
Aurora storage has thousands of
CPUs
Opportunity to push down and parallelize query
processing
Moving processing close to data reduces network
traffic and latency
Challenges
Data is not range partitioned – require full scans
Data may be in-flight
Read views may not allow viewing most recent data
Not all functions can be pushed down
Database Node
Storage nodes
Push down
predicates
Aggregate
results
We were able to test Aurora’s parallel query feature and the performance gains were very good.
To be specific, we were able to reduce the instance type from r3.8xlarge to r3.2xlarge. For this
use case, parallel query was a great win for us.
– Jyoti Shandil, Cloud Data Architect

S U M M I T
Designed to offer >99.99% availability, RPO = 1 second, RTO <1
minute

S U M M I T
Amazon RDS and Aurora High Availability
WAL DATA COMMIT LOG & FILES
Amazon RDS Oracle, SQL Server, PostgreSQL, and MySQL with
MULTI-AZ
EBS mirrorEBS mirror
AZ 1 AZ 2
Amazon S3
EBS
Amazon Elastic
Block Store (EBS)
Primary
Database
Node
Standby
Database
Node
1
2
3
4
5
T Y P E O F W R I T E
AZ 1 AZ 3
Primary
Database
Node
Amazon S3
AZ 2
Read
Replica /
Secondary
Node
AMAZON AURORA
ASYNC
4/6 QUORUM
DISTRIBUTED
WRITES
Read
Replica /
Secondary
Node

S U M M I T
Six-way replicated storage
Survives catastrophic failures
• Six copies across three availability zones
• Four out of 6 write quorum; 3 out of 6
read quorum
• Peer-to-peer replication for repairs
• Volume striped across hundreds of
storage nodes
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching
SQL
Transaction
AZ 1 AZ 2 AZ 3
Caching

S U M M I T
Scalable Read Replicas
• Relieve pressure on your source database with additional read capacity
• Within one region or cross-region
• Promote a Read Replica to a master for faster recovery in the event of
disaster
• Upgrade a Read Replica to a new engine version
• Supported for Amazon Aurora, Oracle, MySQL, MariaDB, and PostgreSQL

S U M M I T
Aurora read replicas
Master
Read
replica
Read
replica
Read
replica
Shared distributed storage volume
Reader endpoint #1 Reader endpoint #2
• Up to 15 promotable read replicas across multiple availability zones
• Re-do log based replication leads to low replica lag – typically < 10ms
• Custom reader end-point with configurable failover order

S U M M I T
Instant crash recovery
Traditional database:
Have to replay logs since the last checkpoint
Typically 5 minutes between checkpoints
Single-threaded in MySQL; requires a large
number of disk accesses
Amazon Aurora:
Underlying storage replays redo records on
demand as part of a disk read
Parallel, distributed, asynchronous
No replay for startup
Checkpointed Data Redo Log
Crash at T0 requires
a re-application of the
SQL in the redo log since
last checkpoint
T0 T0
Crash at T0 will result in redo logs being
applied to each segment on demand, in
parallel, asynchronously

S U M M I T
Database Backtrack
t0 t1 t2
t0 t1
t2
t3 t4
t3
t4
Rewind to t1
Rewind to t3
Invisible Invisible
Backtrack brings the database to a point in time without requiring restore from backups
• Backtracking from an unintentional DML or DDL operation
• Backtrack is not destructive. You can backtrack multiple times to find the right point in time
• Also useful for QA (rewind your DB between test runs)

S U M M I T
Fast database cloning
Production database
Clone Clone
Clone
Dev/test
applications
Benchmarks
Production
applications
Production
applications
Create a copy of a database without
duplicate storage costs
• Creation of a clone is nearly instantaneous –
we don’t copy data
• Data copy happens only on write – when
original and cloned volume data differ
Use cases
• Clone a production DB to run tests
• Reorganize a database
• Save a point in time snapshot for analysis
without impacting production system.

S U M M I T
Scaling write workloads
There are many “oases” of consistency in
Aurora
Database nodes know transaction orders
from that node
Storage nodes know transactions orders
applied at that node
Only have conflicts when data changed at
both multiple database nodes AND multiple
storage nodes
Much less coordination required
Near linear throughput scaling for
workloads with no or low conflict
Lower commit latency for workloads with
low conflict
Optimistic conflict management

S U M M I T
Global replication
Faster disaster recovery and enhanced data locality
• Promote read-replica to a master
for faster recovery in the event of
disaster, RPO = 1 second, RTO <1
minute
• Bring data close to your customer’s
applications in different regions
• Promote to a master for easy
migration

S U M M I T
Aurora Global Database
MR R
REGION 1
AZ 1 AZ 2 AZ 3
SHARED STORAGE
R
REGION 2
AZ 1 AZ 2 AZ 3
SHARED STORAGE
REPLICATION
FLEET
REPLICATION
FLEET
High throughput: Up to 150K writes/sec – negligible performance impact
Low replica lag: < 1 sec cross-country replica lag under heavy load
Fast recovery: < 1 min to accept full read-write workloads after region failure

S U M M I T
Serverless, Data API, and advanced monitoring capabilities

S U M M I T
Aurora Serverless use cases
• Infrequently used applications (e.g., low-
volume blog site)
• Applications with variable load—peaks of
activity that are hard to predict (e.g., news
site)
• Development or test databases not needed
on nights or weekends
• Consolidated fleets of multi-tenant SaaS
applications

S U M M I T
Aurora Serverless architecture
• Starts up on demand,
shuts down when not in
use
• Scales up/down
automatically
• No application impact
when scaling
• Pay per second, 1 minute
minimum
WARM POOL
OF INSTANCES
APPLICATION
DATABASE STORAGE
SCALABLE DB CAPACITY
REQUEST ROUTERS

S U M M I T
Performance Insights
• Dashboard showing
database load
• Easy – e.g., drag and drop
• Powerful – drill down using
zoom-in
• Identifies source of
bottlenecks
• Sort by top SQL
• Slice by host, user, wait events
• Adjustable time frame
• Hour, day, week , month
• Up to 2 years of data; 7 days free
Max vCPU
CPU bottleneck
SQL w/ high CPU

S U M M I T
Amazon RDS Data API
Millions of
IOT/mobile devices Data API fleet
API
End-point
Amazon Aurora
Serverless
Access through simple web interface
• Public endpoint addressable from anywhere
• No client configuration required
• No persistent connections required
Ideal for serverless applications (AWS
Lambda)
Ideal for light-weight applications (IoT)

S U M M I T
Security Management & Certifications

S U M M I T
Security management
• Database activity monitoring
• SQL Server Audit for Amazon RDS for SQL
Server
• Encryption to secure data at rest using
customer managed keys
• AES-256; hardware accelerated
• All blocks on disk and in Amazon Simple Storage
Service (Amazon S3) are encrypted
• Key management via AWS KMS
• Encrypted cross-region replication,
snapshot copy—SSL to secure data in
transit
• Advanced auditing and logging without
any performance impact
Database
Engine
*NEW*
Customer Master Key(s)
Data key 1
Storage
node
Storage
node
Storage
node
Storage
node
Data key 1 Data key 1 Data key 1
*NEW*

S U M M I T
Database activity monitoring
CloudWatch
consumer
Amazon
Aurora
Amazon
CloudWatch
3rd party
consumer
Amazon
Kinesis
Search: Look for specific events across log files
Metrics: Measure activity in your Aurora DB cluster
• Continuously monitor activity in your DB clusters by sending these audit logs to CloudWatch logs
• Export to S3 for long term archival; analyze logs using Amazon Athena; visualize logs with Amazon QuickSight
Visualizations: Create activity dashboards
Alarms: Get notified or take actions

S U M M I T
Industry certifications
• Amazon Aurora gives each
database instance IP firewall
protection
• Aurora offers transparent
encryption at rest and SSL
protection for data in transit
• Amazon VPC lets you isolate and
control network configuration
and connect securely to your IT
infrastructure
• AWS Identity and Access
Management (IAM) provides
resource-level permission
controls
*New*

S U M M I T

S U M M I T
Aurora and Amazon RDS customer adoption

S U M M I T
Pearson
Our Company
We are the world’s learning company with more than
24,000 employees in nearly 60 countries and a mission to
help people make progress in their lives through learning.
We combine world-class educational content and
assessment, powered by services and technology, to
enable more effective teaching and personalized learning
at scale. We believe that wherever learning flourishes so
do people.
Our Vision
Our vision is to have a direct relationship with millions of
lifelong learners and to link education to the way people
aspire to live and work every day. To do that, we’ll
collaborate with a wide group of partners to help shape
the future of learning. We believe that we all need to
embrace lifelong learning, continuously acquiring new
knowledge and skills to thrive in an ever-changing and
increasingly connected world.
“The cloud plays a major role in Pearson’s transformation
into a digital education company. Using AWS, the IT team
created a platform product teams use to create next-
generation apps for education.
AWS enables our teams to release features and update
them globally at any time, lower costs to internal
developer partners, transforming work with security
teams, and delivering value to students. Amazon Aurora
has been a core part of this our AWS infrastructure to
ensure we have best performance, high availability, and
scalability with our increasing users.”
Ian Wright, VP of Infrastructure & Operations - Pearson

S U M M I T
FINRA
FINRA is a not-for-profit organization
dedicated to investor protection and
market integrity. It regulates one
critical part of the securities industry –
brokerage firms doing business with
the public in the United States.
“FINRA has migrated more than 500 relational databases
from on-premises to Amazon RDS PostgreSQL.
Moving to RDS has allowed us to be more agile and
innovative. Our development teams have greater freedom
to experiment and try new things because they can
quickly spin up database instances on demand.
As a next step, we plan to increase our use of Amazon
Aurora to take advantage of the high performance,
scalability, and availability it provides.”
Tigran Khrimian, VP of Data Platforms – FINRA

S U M M I T
Blackboard
Blackboard delivers innovative
education technology and services
that enable millions of people
worldwide to learn in schools,
institutions, and companies.
"Blackboard’s decision to move from on-premises Oracle to
Amazon RDS has allowed the Product Development
organization to refocus DBA resources away from day-to-day
infrastructure work and more towards improving the client
experience.
Removing the daily care and feeding of cross-site replication,
data backups, patching, and hardware health and availability
has really changed how we treat the database.
The ability to quickly scale based on demand, seamlessly
move to newer/faster instance types, and leverage tools to
manage costs in near real-time have provided a level of control
and visibility that the DBAs have never experienced.”
Reinhold Staudinger, Chief Architect – Blackboard

S U M M I T
Arizona State University
Arizona State University, in Tempe,
AZ, is a public research university
ranked #1 ranked by U.S. News &
World Report as the #1 university
in the U.S. for innovation.
"ASU continues to leverage the AWS cloud as a means to
improve our agility and capacity in meeting our customer needs.
One such AWS service we’re bullish on is Amazon Aurora.
We’re moving much of our on premise database workload,
currently MySQL, Sybase, Oracle and MS-SQL Server to Aurora or
RDS. Aurora also plays an integral role in our data lake strategy,
along with Redshift and S3, as we build out our new advanced
analytic platform. Data behind our ASU Alexa skill and ASU
mobile app benefits from Amazon Aurora's scalability and high
performance as enormous concurrent student users during the
school year creates new peak loads each season. And, we see
Amazon Aurora Serverless as a next step in our cloud maturity to
help us improve development agility while reducing costs on
infrequently used systems, to further optimize our overall
infrastructure operations.“
John Rome, Deputy Chief Information Officer - Arizona State
University

S U M M I T
DMS and SCT
Our goal: Allow customers the freedom to choose the best data platform
for their needs #DBFreedom
AWS Schema Conversion Tool (SCT) converts your commercial
database and data warehouse schemas to open-source engines or
AWS-native services, such as Amazon Aurora and Redshift
AWS Database Migration Service (DMS) easily and securely
migrates and/or replicate your databases and data
warehouses to AWS

S U M M I T
AWS DMS – Endpoint Support Expansion
Amazon
Redshift
Amazon
DynamoDB
Amazon S3
Amazon S3
Amazon
Aurora
Amazon
Aurora
Oracle SQL Server Netezza
Greenplum Vertica Teradata
AWS Snowball
Edge
MongoDB Cassandra
Amazon
Elasticsearch
Amazon Kinesis

S U M M I T
Data Migration
Data Lake Foundation
Predictive Analytics
Microservices Deployment
Performance Optimization
3-4 Days Onsite
Architect Solution
Build in your Account
Learn from Experts
Validate Functionality
Data Lab Architect
Solution Architect(2)
DBS Program Managers
DBS Engineers
DBS Specialist SA’s
4-8 Customer Builders
Come with an idea, leave with a solution.

S U M M I T
Amazon Relational Database Service (RDS)
Choice Value Innovation

S U M M I T
Get started!
Amazon Aurora
aws.amazon.com/aurora
AWS Database Migration Service
aws.amazon.com/dms
Amazon RDS
aws.amazon.com/rds/
10-minute tutorials
aws.amazon.com/getting-started/tutorials/?awsf.getting-started-
content=use-case-tmt%23databases

S U M M I T
Thank you!
S U M M I T
Kwesi Edwards
kwesied@amazon.com

S U M M I T
S U M M I T

Scale Up and Modernize Your Database with Amazon Relational Database Service and Aurora

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Scale Up and Modernize Your Database with Amazon Relational Database Service and Aurora

Similar to Scale Up and Modernize Your Database with Amazon Relational Database Service and Aurora (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Scale Up and Modernize Your Database with Amazon Relational Database Service and Aurora