Amazon Aurora with PostgreSQL compatibility is a relational database managed service that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source PostgreSQL. This session highlights Aurora with PostgreSQL compatibility’s key capabilities, including low-latency read replicas and Multi-AZ deployments; reviews the architectural enhancements that contribute to Aurora’s improved scalability, availability, and durability; and digs into the latest feature releases. Finally, this session walks through techniques to migrate to Aurora.
5. Aurora storage and replicas
RW
Application Application
Write log
records
Aurora
storage
Availability Zone 3
Availability Zone 2
Availability Zone 1
6. Aurora storage and replicas
RW
Application Application
Write log
records
Read
blocks
Aurora
storage
Availability Zone 3
Availability Zone 2
Availability Zone 1
7. Aurora storage and replicas
RW
Application Application
Write log
records
Read
blocks
Aurora
storage
Availability Zone 3
Availability Zone 2
Availability Zone 1
8. Aurora storage and replicas
RW
Application Application
Write log
records
Read
blocks
Aurora
storage
Availability Zone 3
Availability Zone 2
Availability Zone 1
9. Aurora storage and replicas
RW
Application Application
Write log
records
Read
blocks
Aurora
storage
Availability Zone 3
Availability Zone 2
Availability Zone 1
10. RO
Application
Aurora storage and replicas
RW
Application
RO
Application
Async
Invalidation
& Update
Async
invalidation
& update
Write log
records
Read
blocks
Aurora
storage
RO
RO
RO
RO
Availability Zone 3
Availability Zone 2
Availability Zone 1
11. RO
Application
Aurora storage and replicas
RW
Application
RO
Application
Async
Invalidation
& Update
Async
invalidation
& update
Write log
records
Read
blocks
Aurora
storage
RO
RO
RO
RO
Availability Zone 3
Availability Zone 2
Availability Zone 1
12. RO
Application
Aurora storage and replicas
RW
Application
RO
Application
Async
Invalidation
& Update
Async
invalidation
& update
Write log
records
Read
blocks
RW
Aurora
storage
RO
RO
RO
RO
Availability Zone 3
Availability Zone 2
Availability Zone 1
13. Availability Zone 2
Availability Zone 1 Availability Zone 3
RO
Application
Fast clones
RW
Application
RW
Reporting
application
Read
blocks
Aurora
storage
Primary storage
Clone storage
Clone
14. Availability Zone 2
Availability Zone 1 Availability Zone 3
RO
Application
Fast clones
RW
Application
RW
Reporting
application
Read
blocks
Aurora
storage
Primary storage
Clone storage
Clone
15. Availability Zone 2
Availability Zone 1 Availability Zone 3
RO
Application
Fast clones
RW
Application
RW
Reporting
application
Read
blocks
Aurora
storage
Primary storage
Clone storage
Clone
16. Availability Zone 2
Availability Zone 1 Availability Zone 3
RO
Application
Fast clones
RW
Application
RW
Reporting
application
Write log
records
Read
blocks
Aurora
storage
Primary storage
Clone storage
Clone
18. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
Queued work
Storage
Transaction
Write
19. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
Queued work
Storage
Transaction
Write
20. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
Queued work
Storage
Transaction
Write
21. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
Queued work
Storage
Transaction
Write
22. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
Queued work
Storage
Transaction
Write
23. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
Queued work
Storage
Transaction
Write
24. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A Queued work
Storage
B
Transaction
Write
25. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A
Queued work
Storage
B C D E
Transaction
Write
26. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A
Queued work
Storage
B C D E
2 2 1 0 1
A B C D E
Durability
tracking
Transaction
Write
27. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A
Queued work
Storage
B C D E
2 2 1 0 1
A B C D E
Durability
tracking
Transaction
Write
G F
28. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A
Queued work
Storage
B C D E
2 2 1 0 1
A B C D E
Durability
tracking
Transaction
Write
G
F
29. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A
Queued work
Storage
B C D E
2 2 1 0 1
A B C D E
Durability
tracking
Transaction
Write
G
F
30. Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A
Queued work
Storage
B C D E
4 3 4 2 4 1
A B C D E F
Durability
tracking
Transaction
Write
G
F
31. 6 5 6 3 5 2 0
A B C D E F G
Concurrency: Remove log buffer
Queued work
Log buffer
PostgreSQL Aurora PostgreSQL
Storage
A
Queued work
Storage
B C D E
Durability
tracking
Transaction
Write
G F
32. Aurora: Writing less
Aurora
Block in
memory
t-v1
Aurora
storage
Block in
memory
PostgreSQL
t-v1
t-v2
WAL
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
33. Aurora: Writing less
Aurora
Block in
memory
t-v1
Aurora
storage
Block in
memory
PostgreSQL
t-v1
t-v2
t-v1
t-v2
Full
block
WAL
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
34. Aurora: Writing less
Aurora
Block in
memory
t-v1
Aurora
storage
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
t-v1
t-v2
Full
block
t-v3
WAL
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
35. Aurora: Writing less
Aurora
Block in
memory
t-v1
Aurora
storage
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
Checkpoint
Datafile
t-v1
t-v2
Full
block
t-v3
WAL
Archive
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
36. Aurora: Writing less
Aurora
Block in
memory
t-v1
Aurora
storage
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
Checkpoint
Datafile
t-v1
t-v2
Full
block
t-v3
WAL
Archive
4K
4K
8K
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
37. Aurora: Writing less
Aurora
Block in
memory
t-v1
Aurora
storage
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
Checkpoint
Datafile
t-v1
t-v2
Full
block
t-v3
WAL
Archive
4K
4K
8K
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
recovery
in minutes
38. Aurora: Writing less
Aurora
update t set y = 6
Block in
memory
t-v1
Aurora
storage
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
Checkpoint
Datafile
t-v1
t-v2
Full
block
t-v3
WAL
Archive
4K
4K
8K
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
recovery
in minutes
39. Aurora: Writing less
Aurora
update t set y = 6
Block in
memory
t-v1
t-v2
Aurora
storage
t-v1
t-v2
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
Checkpoint
Datafile
t-v1
t-v2
Full
block
t-v3
WAL
Archive
4K
4K
8K
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
recovery
in minutes
40. Aurora: Writing less
Aurora
update t set y = 6
Block in
memory
t-v1
t-v2
t-v3
Aurora
storage
t-v1
t-v2
t-v3
No engine
checkpoint
=
no FPW
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
Checkpoint
Datafile
t-v1
t-v2
Full
block
t-v3
WAL
Archive
4K
4K
8K
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
recovery
in minutes
41. Aurora: Writing less
Aurora
update t set y = 6
Block in
memory
t-v1
t-v2
t-v3
Aurora
storage
t-v1
t-v2
t-v3
No engine
checkpoint
=
no FPW
Block in
memory
PostgreSQL
t-v1
t-v2
t-v3
Checkpoint
Datafile
t-v1
t-v2
Full
block
t-v3
WAL
Archive
4K
4K
8K
update t set y = 6
Amazon Simple Storage
Service (Amazon S3)
recovery
in minutes continuous
& parallel
coalesce
recovery in
seconds
66. Partitions in PostgreSQL 12
Partitioned table "public.door_knocks"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+-----------------------------+-----------+----------+---------+----------+--------------+-------------
id | integer | | not null | | plain | |
knock_date | timestamp without time zone | | not null | | plain | |
city | character varying(255) | | not null | | extended | |
zipcode | character(5) | | not null | | extended | |
Partition key: RANGE (knock_date)
Partitions: door_knock_date_y2020103001 FOR VALUES FROM ('2020-10-30 01:00:00') TO ('2020-10-30 02:00:00'),
door_knock_date_y2020103002 FOR VALUES FROM ('2020-10-30 02:00:00') TO ('2020-10-30 03:00:00'),
door_knock_date_y2020103003 FOR VALUES FROM ('2020-10-30 03:00:00') TO ('2020-10-30 04:00:00')
68. Storage Management – dynamic resizing
new partitions
every hour
drop
existing
create
new
69. Storage Management – dynamic resizing
new partitions
every hour
drop
existing
create
new
2 hour
spike
70. Storage Management – dynamic resizing
new partitions
every hour
drop
existing
create
new
2 hour
spike drop
existing
create
new
71. Storage Management – dynamic resizing
new partitions
every hour
drop
existing
create
new
2 hour
spike drop
existing
create
new
drop
the
spike
72. Storage Management – dynamic resizing
new partitions
every hour
drop
existing
create
new
2 hour
spike drop
existing
create
new
drop
the
spike
used space
inside the db
used storage
space
2X extra
storage
costs
73. Storage Management – dynamic resizing
new partitions
every hour
drop
existing
create
new
2 hour
spike drop
existing
create
new
drop
the
spike
used space
inside the db
used storage
space
85. 12–300 ms
Cross region replicas: PostgreSQL
PostgreSQL
RW
EBS
PostgreSQL
RO
EBS
Update
Region A Region B
86. 12–300 ms
Cross region replicas: PostgreSQL
PostgreSQL
RW
EBS
PostgreSQL
RO
EBS
Update
Extra expense
Region A Region B
87. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
88. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
89. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
90. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
91. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
92. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
DR
93. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
RO
94. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
RO
95. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
RO
Application Application
RO
Application
RO
96. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
RO
Application Application
RO
Application
RO
97. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
RO
Application Application
RO
Application
RO
98. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
RO
Application Application
RO
Application
RO
99. Replication
agents
Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers Aurora storage
RO
Application Application
RO
Application
RO
100. Region B
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Availability Zone 3
Availability Zone 1 Availability Zone 2
Amazon Aurora Global Database
Aurora storage
RO
Application
RW
Application
RO
Application
Aurora storage
RO
Application Application
RO
Application
RO
RW
101. Amazon Aurora Global Database
Region A
Availability Zone 3
Availability Zone 1 Availability Zone 2
Aurora storage
RO
Application
RW
Application
RO
Application
Replication
servers
Region B
Availability Zone 3
Availability Zone 1 Availability Zone 2
Replication
agents Aurora storage
R
O
Application Application
R
O
Application
R
O
Region C
Availability Zone 3
Availability Zone 1 Availability Zone 2
Replication
agents Aurora storage
R
O
Application Application
R
O
Region D
Availability Zone 3
Availability Zone 1 Availability Zone 2
Replication
agents Aurora storage
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 128 TB
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Quorum system for read/write; latency tolerant
Quorum membership changes do not stall writes
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 64 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 64 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 64 TB
Data is replicated 6 times across 3 Availability Zones
Continuous backup to Amazon S3
Continuous monitoring of nodes and disks for repair
10GB segments as unit of repair or hotspot rebalance
Storage volume automatically grows up to 64 TB
Sysbench 10K rows with 250 tables
Sysbench 300K rows with 250 tables
AWS Database Migration Service (AWS DMS) – heterogeneous or homogeneous and support different versions
PostgreSQL: pg_dump/pg_restore – PostgreSQL only but can move to higher versions
PostgreSQL Logical Replication – PostgreSQL only but can move to higher versions but needs to be version 9.4+
Amazon RDS PostgreSQL: Snapshot import – Useful for smaller or test instances on RDS PostgreSQL with same version
Amazon RDS PostgreSQL: Read replica – Simple and smaller downtime if using RDS PostgreSQLwith same version