DOWNLOAD SO ANIMATIONS WORK PROPERLY. THANK YOU! Amazon Aurora is a cloud relational database built from the ground up with a new, ingenious architecture. This video is part of a series. This Section 2.0 of Amazon Aurora's Deep Dive, primarily on its architecture, has 9 videos to compliment this slide deck. It begins at https://youtu.be/Cnz6mSzca1Y . In this Section I cover at a deeper level architecture from Section 1.0 of my Amazon Aurora Deep Dive videos as well as many more amazing feature innovations. In Section 2.0 here I dive deep into the Amazon Aurora Database Cluster architecture & how that works, another sample of Aurora Clusters using Replicas as well as Cross-Region Read Replicas, it compares Amazon Aurora to MySQL for read scaling, it covers Aurora's Endpoints, & how Aurora's Replication compares to MySQL insofar as lag time is concerned. I then dive DEEP into Amazon Aurora's Logging, Storage, Caching, & Indexing & how that's accomplished. It covers log-structured storage, B-tree indexes, & Aurora's garbage collection. It teaches you how Aurora accomplishes Instant Crash Recovery, Survivable Caches, & then I dive DEEP again into Amazon Aurora's Input/Output (IO) architecture (you'll find this amazing!). I cover how Aurora has fewer IOs/Second, how Network-Attached Storage optimizes Packets/Second, & how Aurora has more asynchronous processing. I compare IO traffic in RDS MySQL to BOTH IO traffic in an Aurora DATABASE as well as IO traffic in its storage nodes. I compare traditional database commits to Aurora's asynchronous group commits, & Aurora's creative Adaptive Thread Pool. I compare IO traffic in read replicas of MySQL vs. Amazon Aurora, then discuss how to understand Aurora's scalability (up or out) & elasticity. You'll learn about Aurora's fault tolerance & restoration architecture & how it accomplishes fast, predictable failovers, I cover Amazon Aurora's backup & restore features, read replicas priority tiers, & how to simulate failures using SQL for testing. I teach how Aurora's performance is blazingly fast via enhancements, & then I go over benchmarking. Lastly, I cover Aurora's security architecture. NOTE: this presentation was made a couple years ago, but you'll have a great foundation from which to progress in your understanding

1. Amazon Aurora Deep Dive
Video 2.0: Amazon Aurora’s Architecture
“Invent and Simplify”
By Kim Schmidt
Amazon RDS
https://www.amazon.jobs/principles *Image courtesy of AWS properties

2. Section 2.1: Amazon Aurora’s Architecture
“The Unique Architecture of Amazon Aurora, created by re-designing the RDB for TODAY where AWS exists”
*Image courtesy of AWS properties

3. Architectural High-Level View from Video Chapter 1.0: Video Section 1.3
https://youtu.be/Cnz6mSzca1Y?list=PL2EgRNBrbhC4lbTLy9cZnHG2JvEd8wZCr
*Image courtesy of AWS properties

4. “Simple” Representation of an Amazon Aurora Cluster
*Image courtesy of AWS properties

5. “Simple” Representation of an Amazon Aurora Cluster with Replicas
*Image courtesy of AWS properties

6. Amazon Aurora’s Read Scaling vs. MySQL Read Scaling
*Image courtesy of AWS properties

7. Now Let’s Dig Deeper Into Each Architectural Topic

8. For wha
Quiz Time for Section 2.1: Amazon Aurora’s Unique Architecture
Q: Which of the following is NOT true about Amazon Aurora’s Unique Architecture?
a) An Amazon Aurora cluster volume is SSD-based
b) Amazon Aurora’s storage latency can cause data drift between the Primary Instance and the Read Replicas
c) An Amazon Aurora cluster volume spans 3 Availability Zones with 2 storage nodes per Availability Zone, totaling 6 nodes
d) Amazon Aurora uses a service-oriented architecture applied to a database for storage and logging
A: b) Amazon Aurora’s storage latency can cause data drift between
the Primary Instance and the Read Replicas
Explanation:
MySQL’s independent storage layers that have to be monitored & managed
and latency can cause data drift between the Master & the Replica
depending on how much traffic goes thorough. Amazon Aurora’s storage is
shared, so the Replicas can focus 100% on Reads, not Writes

9. This Concludes
Section 2.1: Amazon Aurora’s Unique Architecture
Coming Up Next is Section 2.2:
Amazon Aurora’s Clusters and Replicas

10. Section 2.2: Amazon Aurora’s Clusters and
Replicas
By Kim Schmidt

11. Creating an Amazon Aurora Cross-Region Read Replica - 1
*Image courtesy of AWS properties

12. Creating an Amazon Aurora Cross-Region Read Replica - 2

13. Creating an Amazon Aurora Cross-Region Read Replica - 3

14. Creating an Amazon Aurora Cross-Region Read Replica - 4

15. Creating an Amazon Aurora Cross-Region Read Replica - 5

16. Creating an Amazon Aurora Cross-Region Read Replica - 6

17. For wha
Quiz Time for Section 2.2: Amazon Aurora’s Clusters and Read Replicas
Q: Which of the following is NOT true about Amazon Aurora’s Clusters and Read Replicas?
a) For Cross-Region Replication, you need to ensure that the VPC and the DB Subnet Groups exist in the target region
b) Multiple Read Replicas can be placed in the same Availability Zone
c) As of October, 2016, Amazon Aurora is available in 7 Regions
d) Ensure binary logging for the Parameter Group you’re launching a Cross-Region Aurora cluster is set to “STATEMENT”
A: d) Ensure binary logging for the Parameter Group you’re launching
a Cross-Region Aurora cluster is set to “STATEMENT”
Explanation:
For Cross-Region Replication, he Parameter Group’s binary logging should
be set to “MIXED”. The reason for this is Amazon Aurora cross-region
replication uses MySQL binary replication to replay changes on the cross-
region Read Replica DB cluster.

18. This Concludes
Section 2.2: Amazon Aurora’s Clusters
and Replicas
Coming Up Next is Section 2.3:
Amazon Aurora’s Endpoints and Replication

19. Section 2.3: Amazon Aurora’s Endpoints
and Replication
By Kim Schmidt

20. Amazon Aurora’s Endpoints - 1

21. Amazon Aurora’s Endpoints - 2

22. Amazon Aurora’s Endpoints - 3

23. Amazon Aurora’s Endpoints - 4

24. Amazon Aurora’s Replication
*Image courtesy of AWS properties

25. For wha
Quiz Time for Section 2.3: Amazon Aurora’s Endpoints and Replication
Q: What is the Recommended and Safest Endpoint to Connect to Your Database Cluster?
a) Instance Endpoints because they allow direct connection to the Replicas
b) Reader Endpoints to be safe during a failover
c) Reader Endpoints to ensure load-balancing across Read Replicas
d) Cluster Endpoints because your application will reconnect to the new Primary Instance once failover is complete
Answer: d) Cluster Endpoints because your application will reconnect
to the new Primary Instance once failover is complete
Explanation:
During a failover, Aurora continues to serve requests to the Cluster
Endpoint from the newly promoted Primary Instance as that replaces the
failed instance. Aurora serves the cluster endpoint with minimal interruption
of service.

26. This Concludes
Section 2.3: Understanding Amazon Aurora’s
Endpoints and Replication
Coming Up Next is Section 2.4:
Understanding Amazon Aurora’s Logging, Storage,
Caching, and Indexing

27. Section 2.4: Understanding Amazon Aurora’s
Logging, Storage, Caching, and Indexing
By Kim Schmidt

28. Amazon Aurora’s Log-Structured Storage - 1
*Image courtesy of AWS properties

29. Amazon Aurora’s Log-Structured Storage - 2
*Image courtesy of AWS properties

30. Amazon Aurora’s Log-Structured Storage - 3
 Records are APPENDED to storage –
existing records are never updated
 B-Tree indexes hold pointers to the latest
version of a record
 This means you’re going to have stale
data, that’s periodically removed through
a garbage collection process

31. Amazon Aurora’s Log-Structured Storage - 4
*Image courtesy of AWS properties

32. Some Benefits of Amazon Aurora’s Log-Structured Storage – (5)
*Image courtesy of AWS properties

33. Some Benefits of Amazon Aurora’s Log-Structured Storage – (6)
*Image courtesy of AWS properties

34. For wha
Quiz Time for Section 2.4: Understanding Amazon Aurora’s
Logging, Storage, Caching & Indexing
Q: What is NOT true about Amazon Aurora’s Logging, Storage, Caching, & Indexing?
a) B Trees don’t have primary keys; they have corresponding leaves
b) It has a garbage collection mechanism
c) The database itself is it’s own write-ahead log
d) Caching is outside the database process and can survive a database restart
A: a) B Trees don’t have primary keys; they have corresponding
leaves
Explanation:
B Trees do have primary keys. They point to other sections that eventually
point to leaf nodes.

35. This Concludes Section 2.4: Understanding
Amazon Aurora’s Logging, Storage,
Caching, and Indexing
Coming Up Next is Section 2.5:
Understanding Amazon Aurora’s I/O

36. M
Section 2.5: Understanding Amazon Aurora’s
I/O Architecture
By Kim Schmidt

37. u
Amazon Aurora’s Unique I/O

38. IO Traffic in RDS MySQL
*Image courtesy of AWS properties

39. IO Traffic in an Amazon Aurora Database
*Image courtesy of AWS properties

40. IO Traffic in an Amazon Aurora Storage Node
*Image courtesy of AWS properties

41. Amazon Aurora’s Asynchronous Group Commits
*Image courtesy of AWS properties

42. Amazon Aurora’s Adaptive Thread Pool
*Image courtesy of AWS properties

43. I/O Traffic in Amazon Aurora’s Read Replicas
*Image courtesy of AWS properties

44. For wha
Quiz Time for Section 2.5: Understanding Amazon Aurora’s I/Os
Q: What isn’t a way that Amazon Aurora has improved I/O?
a) I/O Traffic is improved by sending smaller packets from the Primary Instance to the Replicas
b) Amazon Aurora storage nodes works synchronously from the peer-to-peer gossip stage, to get the heavy work done upfront
c) I/O is improved through the way database commits occur
d) I/O is improved through connection threading via epoll()
A: b) Amazon Aurora storage nodes works synchronously from the
peer-to-peer gossip stage, to get the heavy work done upfront
Explanation:
For each node, the synchronous work is done once the update queue
acknowledges back to the Primary Instance that things are done, the
remaining steps are asynchronous, including the peer-to-peer gossiping

45. This Concludes Section 2.5: Understanding
Amazon Aurora’s I/O
Coming Up Next is Section 2.6: Understanding
Amazon Aurora’s Scalability and Elasticity
Architectures

46. Section 2.6: Understanding Amazon Aurora’s
Scalability and Elasticity Architecture
By Kim Schmidt

47. Amazon Aurora’s Scalability vs. Elasticity

48. Relational Databases Inherently are Difficult to Scale
*Image courtesy of AWS properties

49. Amazon Aurora’s Elasticity

50. For wha
Quiz Time for Section 2.6: Understanding Amazon Aurora’s
Scalability and Elasticity Architectures
Q: What is the BEST takeaway from this short chapter on Amazon Aurora’s Scalability and Elasticity?
a) Scalability is NOT Elasticity
b) Elasticity is defined as the degree to which a system is able to adapt to workload changes
c) The goal of being scalable is to be able to be available to your customers as demand for your application grows.
d) Amazon Aurora auto-scales up to meet any peak in traffic and then auto-scales down when demand for your application is less,
providing both great scalability and elasticity.
A: d) Amazon Aurora auto-scales up to meet any peak in traffic then
down when demand for your application is less, providing both great
scalability and elasticity.
Explanation:
Amazon Aurora is the first truly scalable and elastic Relational Database
System, solving decades of limitations and frustrations, and is the best
answer in regard to specific question about Amazon Aurora’s scalability &
elasticity; whereas answer “A” is a generalized answer

51. This Concludes Section 2.6: Understanding
Amazon Aurora’s Scalability and Elasticity
Architectures
Coming Up Next is Section 2.7: Understanding
Amazon Aurora’s Fault Tolerance
and Restoration Architecture

52. Section 2.7: Understanding Amazon Aurora’s
Fault Tolerance and Restoration Architecture
By Kim Schmidt

53. Amazon Aurora’s Fault Tolerance
*Image courtesy of AWS properties

54. Amazon Aurora’s Instant Crash Recovery
*Image courtesy of AWS properties

55. Amazon Aurora’s Fast, More Predictable Failover
*Image courtesy of AWS properties

56. And Remember…
*Image courtesy of AWS properties

57. Amazon Aurora’s Backup and Restore - 1
 Automated backups are always enabled on Amazon S3
 You can take database snapshots at any time with no performance impact
 If the database fails, Amazon Aurora automatically attempts to recover
your database in a healthy AZ with no data loss
 In the unlikely event that your data is unavailable within Amazon Aurora
storage, you can restore from a database snapshot or perform a “point-in-
time” restore operation to a new instance
 You can share snapshots with a different AWS account, upon which the
recipient can use the snapshot to restore the database, useful when
sharing the same database between various environments (production,
dev/testing, staging, etc)
 You can choose to make your snapshots public
 You can share manual snapshots with up to 20 AWS account IDs. Beyond
that, you have a choice to make the snapshot public or request an
increase in your quota
 You can share snapshots in all Regions where Amazon Aurora is available
– however, the sharing must be within the same Region as the account
that shared it

58. Amazon Aurora’s Backup and Restore - 2
 Amazon Aurora supports 2 kinds of replicas, Read Replicas and by
creating a MySQL Read Replica based on MySQL’s binlog- based
replication engine
 You can set up cross-region Aurora Replicas, and you can add
Replicas to that Cross-Region Replica
 You can choose to promote the Cross-Region Replica to be the new
Primary Instance; however thereafter the pre-existing Cross-Region
Replication will no longer exist
 You can prioritize certain replicas as failover targets by assigning a
Priority Tier – contention chooses the Replica the same size as the
Primary
 You can change modify Priority Tiers any time without triggering a failover
 You can prevent certain Replicas from being promoted to the Primary
Instance by assigning lower Priority Tiers (0-15, 0=highest)
 You can recover your data by creating a new Aurora DB cluster from the
backup data that Aurora retains, or from a DB snapshot you saved

59. Amazon Aurora’s Read Replica Priority Tiers
*Image courtesy of AWS properties

60. Simulating Failures Using SQL - 1
 To force crash an Amazon Aurora instance, use the ALTER SYSTEM CRASH:
ALTER SYSTEM CRASH [ {INSTANCE | DISPATCHER | NODE} ] ;
 To test disk congestion, simulating disk failure for an Aurora database cluster, use ALTER SYSTEM
SIMULATE DISK CONGESTION:
ALTER SYSTEM SIMULATE percentage_of_failure PERCENT
DISK CONGESTION BETWEEN minimum AND maximum MILLISECONDS
[ IN DISK index | NODE index }
FOR INTERVAL quantity [ YEAR | QUARTER | MONTH | WEEK | DAY | HOUR | MINUTE | SECOND];

61. Simulating Failures Using SQL - 2
 To simulate a disk failure for an Aurora database cluster, use the ALTER SYSTEM SIMULATE DISK
FAILURE:
ALTER SYSTEM SIMULATE percentage_of_failure PERCENT DISK FAILURE
[ IN DISK index | NODE index ]
FOR INTERVAL quantity [ YEAR | QUARTER | MONTH | WEEK | DAY | HOUR | MINUTE | SECOND];
 To test an Aurora Replica failure, use ALTER SYSTEM SIMULATE READ REPLICA FAILURE:
ALTER SYSTEM SIMULATE percentage_of_failure PERCENT
READ REPLICA FAILURE [ TO ALL | TO ‘replica name” ]
FOR INTERVAL quantity [ YEAR | QUARTER | MONTH | WEEK | DAY | HOUR | MINUTE | SECOND];

62. Restoring Data - 1
 Create new Aurora DB cluster from backup data Aurora retains
 From a manual snapshot
 To determine the latest or earliest restorable time for a DB instance,
look for Latest Restorable Time or Earliest Restorable Time values on the
RDS console. The latest restorable time for a DB cluster is the most
recent point where you can restore your cluster from, typically 5
minutes of the current time. The earliest restorable time specifies
how far back within the backup retention period you can restore your
cluster volume
 You can determine when the restore of a DB cluster is complete by
checking the Latest Restorable Time or Earliest Restorable Time. Both
values will return NULL until the restore operation is complete.

63. Restoring Data - 2

64. Restoring Data – 2, continued

65. For wha
Quiz Time for Section 2.7: Understanding Amazon Aurora’s
Fault Tolerance and Restoration Architectures
Q: What is NOT true about Amazon Aurora’s Fault Tolerance and Restoration?
a) You can simulate failure of any number of Amazon Aurora’s Read Replicas simultaneously
b) The lowest numbered Priority Tier will be promoted to the Primary Instance upon failure
c) The MariaDB ODBC/JDBC drivers automates the process of DNS propagation to the new Primary Instance upon failure
d) You can share Amazon Aurora snapshots among different AWS Accounts
A: a) You can simulate failures of any number of Amazon Aurora’s
Read Replicas simultaneously
Explanation:
You can only simulate failures on “all” Read Replicas or “one” individual
Read Replica at a time

66. This Concludes Section 2.7: Understanding
Amazon Aurora’s
Fault Tolerance and Restoration Architecture
Coming Up Next is Section 2.8:
Amazon Aurora’s Performance

67. Section 2.8: Understanding Amazon Aurora’s
Performance
By Kim Schmidt *Image courtesy of AWS properties

68. Amazon Aurora’s Performance Enhancements
*Image courtesy of AWS properties
 “Fast Insert”: LOAD DATA and INSERT INTO…SELECT…
 You can monitor the following metrics to determine the effectiveness of “fast insert” for
your database cluster:
• “aurora_fast_insert_cache_hits”: A counter that’s incremented when the cached cursor
is successfully retrieved and verified
• “aurora_fast_insert_cache_misses”: A counter that’s incremented when the ca hed
cursor is no longer valid & Aurora performs a normal index traversal
 You can retrieve the current value of the fast insert metrics using this command:
mysql> show global status like ‘Aurora_fast_insert%’;
 You’ll get an output similar to the following:
+---------------------------------------------------+----------------------+
| Variable _name | Value |
+---------------------------------------------------+----------------------+
| Aurora_fast_insert_cache_hits | 3597400 |
| Aurora_fast_insert_cache_misses | 436494748 |
+---------------------------------------------------+----------------------+

69. This Concludes Section 2.8: Understanding
Amazon Aurora’s Performance
Coming Up Next is Section 2.9:
Amazon Aurora’s Benchmarking

70. Section 2.9: Amazon Aurora’s
Benchmarking
By Kim Schmidt

71. MySQL Sysbench Write and Read Performance Benchmarks
*Image courtesy of AWS properties

72. To Reproduce These Results…
*Image courtesy of AWS properties
https://d0.awsstatic.com/product-
marketing/Aurora/RDS_Aurora_Performance_Assessment_
Benchmarking_v1-2.pdf

73. “If Only” Real-World” Applications Saw Benchmark Performance…

74. “Real World” Scaling User Connections
*Image courtesy of AWS properties

75. “Real World” Scaling Table Count
*Image courtesy of AWS properties

76. “Real World” Scaling Dataset Size
*Image courtesy of AWS properties

77. “Real World” Experiment with Aurora Read Replicas
*Image courtesy of AWS properties

78. How Are These Amazing Results Achieved?
*Image courtesy of AWS properties

79. New Amazon Aurora Update from Jeff Barr’s Blog
 Lambda Function Invocation – The stored procedures you create within you Amazon Aurora databases
can now invoke AWS Lambda functions
 Load Data From S3 – You can now import data stored in an Amazon S3 bucket into a table in an
Amazon Aurora database
https://aws.amazon.com/blogs/aws/
https://aws.amazon.com/blogs/aws/amazo
n-aurora-update-call-lambda-functions-
from-stored-procedures-load-data-from-s3/
Jeff Barr

80. For wha
Quiz Time for Section 2.9: Amazon Aurora’s
Benchmarking
Q: Which statement below BEST summarizes how Amazon Aurora’s Amazing Performance is
Achieved?
1. The Aurora Team architected Amazon Aurora to achieve 107K write/sec, and 585K reads/sec!
a) The Aurora Team architected Amazon Aurora to achieve a higher number of network packet/sec without affecting performance
b) The Aurora Team architected Amazon Aurora to do less work more efficiently
a) The Aurora Team architected Amazon Aurora so that all data fits in both the data dictionary and buffer cache!
A: b) The Aurora Team architected Amazon Aurora to do less work
more efficiently
Explanation:
In essence, the Amazon Aurora team reduced IO in as many places as
much as they could to provide incredible, predictable performance

81. This Concludes Section 2.9: Amazon Aurora’s
Benchmarking
Coming Up Next is the Last Section of 2.0,
Section 2.10: Understanding Amazon Aurora’s
Security Architecture

82. Section 2.10: Understanding Amazon Aurora’s
Security Architecture
By Kim Schmidt

83. AWS and YOU Share Responsibility for Security
*Image courtesy of AWS properties
https://
aws.a
mazon
.com/
market
place/

84. Aurora Database Clusters MUST Be Created in a VPC
http://docs.aws.amaz
on.com/AmazonRDS
/latest/UserGuide/CH
AP_SettingUp.html

85. Virtual Private Clouds
*Image courtesy of AWS properties
Virtual Private Cloud

86. This Concludes
Section 2.10: Understanding
Amazon Aurora’s Security Architecture
And the 2.0 Video “Amazon Aurora’s Architecture”
Coming Up Next is
Section 3.0: Amazon Aurora’s
Configuration and Management,
In the Next Video
I ran out of time before I could finish any more Sections. Apologies!