Devsumi2019 dynamodb

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon DynamoDB
2019/02/14

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• ( ) id:oranie
•
–Web IP
– Web MySQL
Cassandra
–Cassandra summit JPN 2014, 2017
•
–NoSQL DynamoDB

Users 1 million+
Data volume TB, PB, EB
Locality Global
Performance Milliseconds, microseconds
Request rate Millions
Access Mobile, IoT, devices
Scale Up and down
Economics Pay as you go
Developer access Instant API access

• D
• B
• DB
1
B
• BD B

1 L
•
r i H r
• e M Q
• S W Q
• H 1 L y t

RDS for MySQL
B
a D
I A
I A
e a D
I AM
D
Amazon RDS Amazon RDS
AWS Cloud
VPC
Amazon EC2 Amazon EC2
Auto Scaling group

EC
Amazon.com
— Werner Vogels
CTO, Amazon

Amazon DynamoDB
Fully managed nonrelational database for any scale
Fast, consistent performance
Virtually unlimited throughput
Virtually unlimited storage
Encryption at rest and transit
Fine-grained access control
PCI, HIPAA, FIPS140-2 eligible
Service-level agreement
Maintenance free
Serverless
Auto scaling
Backup and restore
Global tables

DynamoDB
Advancements over the last 21 months
VPC
endpoints
April 2017
Auto
scaling
June 2017
DynamoDB
Accelerator (DAX)
April 2017
Time To
Live (TTL)
February 2017
Global tables
On-demand
backup
Encryption at rest
November 2017 November 2017 November 2017
Point-in-time
recovery
March 2018
SLA
June 2018
99.999% SLA
August 2018
Adaptive
capacity ACID
November 2018
Transactions
November 2018
On-demand

Fully managed
Maintenance free
Security
Operating system patching
Database patching
Access control
Audit
Encryption
Compliance
Durability
Sustain server, rack, and datacenter outages
Re-replicate data quickly upon hardware failure
Manage backup and restore
Availability
High availability configuration
Monitoring
Cross-region replication
Performance
Performance tuning
Indexing
In-memory caching
Scalability
Capacity planning
Host provisioning
Host repair and retirement

You build it, you run it.

table
items
attributes
Partation
Key
Sort Key
1:N
Query
==, <, >, >=, <=
“begins with”
“between”
sorted results
counts
/ N
name/value JSON

00 55 A954 FFAA00 FF
Partition Keys
Id = 1
Name = Jim
Hash (1) = 7B
Id = 2
Name = Andy
Dept = Eng
Hash (2) = 48
Id = 3
Name = Kim
Dept = Ops
Hash (3) = CD
Key Space

Partition 3
Partition:Sort Key
: Item
(∞)
§
00:0 FF:∞
Hash (2) = 48
Customer# = 2
Order# = 10
Item = Pen
Customer# = 2
Order# = 11
Item = Shoes
Customer# = 1
Order# = 10
Item = Toy
Customer# = 1
Order# = 11
Item = Boots
Hash (1) = 7B
Customer# = 3
Order# = 10
Item = Book
Customer# = 3
Order# = 11
Item = Paper
Hash (3) = CD
55 A9:∞54:∞ AAPartition 1 Partition 2

Partitions are three-way replicated
Id = 2
Name = Andy
Dept = Engg
Id = 3
Name = Kim
Dept = Ops
Id = 1
Name = Jim
Id = 2
Name = Andy
Dept = Engg
Id = 3
Name = Kim
Dept = Ops
Id = 1
Name = Jim
Id = 2
Name = Andy
Dept = Engg
Id = 3
Name = Kim
Dept = Ops
Id = 1
Name = Jim
Replica 1
Replica 2
Replica 3

A,D B,E C,F
Parition1 Partition2 ParitionN
Partation Key
1
RCU 3000 or WCU 1000
: x
x/N x/N x/N

Adaptive Throughput

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Adaptive Throughput Kicks In
Helped more than 2,800 Amazon tables and indexes on
Prime Day!

Local Secondary Index (LSI)
• 0 a G 0G
• 0 1
• B e SP G1b
A1
(PK)
A3
(Sort)
A2
(table key)
A1
(PK)
A2
(Sort)
A3 A4 A5
LSIs A1
(PK)
A4
(Sort)
A2
(table key)
A3
(projected)
Table
KEYS_ONLY
INCLUDE A3
A1
(PK)
A5
(Sort)
A2
(table key)
A3
(projected)
A4
(projected) ALL

LSI
LSI
Query
()

Global Secondary Index (GSI)
Partition Key
Partition Key
A1
(PK)
A2 A3 A4 A5
GSIs A5
(PK)
A4
(Sort)
A1
(table key)
A3
(projected)
Table
INCLUDE A3
A4
(PK)
A5
(Sort)
A1
(table key)
A2
(projected)
A3
(projected) ALL
A2
(PK)
A1
(table key)
KEYS_ONLY

GSI
GSI
GSI
)
(
K IK
K GK
F
(
IK K
GK K
F
(
K IK
K GK

GSI
Table
Primary
table
Primary
table
Primary
table
Primary
table
Global
Secondary
Index
Client
1. Update request
2. Asynchronous
update (in progress)
2. Update response
GSI !

SQL vs. NoSQL design pattern
Product Database
ID
Type
Price
Description
Products
ID
Title
Category
Fiction
Producer
Director
…
Videos
ID
Artist
Title
Genre
…
Albums
ID
Author
Title
Fiction
Category
Date
…
Books
{
Product ID,
Type,
Price,
Description,
Author,
Title,
Fiction,
Category,
Date,
…
}
ID
AlbumID
Title
Duration
…
Tracks
ActorID
VideoID
ActorVideo
ID
Name
Age
Gender
Bio
…
Actors
{
Product ID,
Type,
Price,
Description,
Artist,
Title,
Genre,
Tracks: [ {
Title1,
Duration1
},{
Title2,
Duration2,
} ]
…
}
{
Product ID,
Type,
Price,
Description,
Title,
Category,
Fiction,
Producer,
Director,
Actors: [ {
ActorID,
Name,
Age,
Gender,
ShortBio
}, … ]
}
Normalization Aggregation

: 1 A
• 0 A @ D I PE J-1.
• - . 5:- . 0.
• 2B A. , = SG U
1 -
2B A. @ , = @ = 5@ @
2B A. A , = A D :@@ 5@ @
1 -
, = @ = 5@ 2B A. @ @
, = A D :@@ 5@ 2B A. A @

- 1
• 3 D 1 D Pa I 1,
• 0 D ,Pa KQS
• c U bN P HKG
3 - D 1 D A
2A , 3= , 3= :15
15 3
2A , 3= , 3= , :15
15 3
2A , 3= , 3= :15
15 3

:
• , 3 , 1 1 2 2
G
• : =
• I P SG I PA SGQ I
P = U
- -
, ,
, ,
, ,
- -
, ,
, ,
, ,

https://www.qwiklabs.com

Targeting queries
Query filters, composite keys, and sparse indexes

Secondary index
Opponent Date GameId Status Host
Alice 2014-10-02 d9bl3 DONE David
Carol 2014-10-08 o2pnb IN_PROGRESS Bob
Bob 2014-09-30 72f49 PENDING Alice
Bob 2014-10-03 b932s PENDING Carol
Bob 2014-10-03 ef9ca IN_PROGRESS David
BobPartition key Sort key

Secondary Index
Approach 1: Query filter(Filter )
Bob
Opponent Date GameId Status Host
Alice 2014-10-
02
d9bl3 DONE Davi
d
Carol 2014-10-
08
o2pnb IN_PROGRESS Bob
Bob 2014-09-
30
72f49 PENDING Alice
Bob 2014-10-
03
b932s PENDING Carol
Bob 2014-10-
03
ef9ca IN_PROGRESS Davi
d
SELECT * FROM Game
WHERE Opponent='Bob'
ORDER BY Date DESC
FILTER ON Status='PENDING'
(filtered out)

( ) 2
StatusDate
DONE_2014-10-02
IN_PROGRESS_2014-10-08
IN_PROGRESS_2014-10-03
PENDING_2014-09-30
PENDING_2014-10-03
Status
DONE
IN_PROGRESS
IN_PROGRESS
PENDING
PENDING
Date
2014-10-
02
2014-10-
08
2014-10-
03
2014-10-
03
2014-09-
30
+ =

Secondary Index
( ) 2
Opponent StatusDate GameId Host
Alice DONE_2014-10-02 d9bl3 David
Carol IN_PROGRESS_2014-10-08 o2pnb Bob
Bob IN_PROGRESS_2014-10-03 ef9ca David
Bob PENDING_2014-09-30 72f49 Alice
Bob PENDING_2014-10-03 b932s Carol
Partition key Sort key

Opponent StatusDate GameId Host
Alice DONE_2014-10-02 d9bl3 David
Carol IN_PROGRESS_2014-10-08 o2pnb Bob
Bob IN_PROGRESS_2014-10-03 ef9ca David
Bob PENDING_2014-09-30 72f49 Alice
Bob PENDING_2014-10-03 b932s Carol
Secondary index
( ) 2
Bob
SELECT * FROM Game
WHERE Opponent='Bob'
AND StatusDate BEGINS_WITH 'PENDING'

Sparse indexes
Id
(Partition)
User Game Score Date Award
1 Bob G1 1300 2012-12-23
2 Bob G1 1450 2012-12-23
3 Jay G1 1600 2012-12-24
4 Mary G1 2000 2012-10-24 Champ
5 Ryan G2 123 2012-03-10
6 Jones G2 345 2012-03-20
-
Award
(Partition)
Id User Score
Champ 4 Mary 2000
Scan sparse GSIs
,

• ) ( ( ) ( (
A A S K C
• ( ) ( ( A
S
Status + Date

4/ 25 3 2 /1 4/
• ( G b O S B ID
• 2 O S D G 0 2 D a
• G (D G D D e c ) 1
• ) 1 0 Gd 0 2G d
n : D EO : V
n L o S
IN i ( G NR687 ) n /
A A

B
IC
13
A
K M
D
S
PR
2 1
1
2 1
1
2 1
1
2 3

G
C
1
1 1
A
1
E
ID

) L
E
1( (
PC
D
GO
I
N A
K
E
PC
N A
K
Partition Key Sort Key
D
GSI

( ) (
• I D B
So
• I D
Ø D a M
n D G M m R

(
)

5 0 1 12C
Shoppers
70,000/sec
SELECT Id, Description, ...
FROM ProductCatalog
WHERE Id="POPULAR_PRODUCT"

https://aws.amazon.com/jp/blogs/database/how-to-use-the-new-amazon-dynamodb-key-diagnostics-library-to-
visualize-and-understand-your-applications-traffic-patterns/
How to use the new Amazon DynamoDB key diagnostics library to visualize and
understand your application’s traffic patterns

DynamoDB
100 /rps
DynamoDB API compatible
Your applications
DAX
DynamoDB
High performance
Amazon DynamoDB Accelerator (DAX)

Write sharding
Handling high velocity writes

, / /10 , / /10 , / /10 , / /10
2, 2,
5 7 C7
0 ) (

• e D S
• W
S
• aQ iB
Amazon DynamoDB
Instances
Instance
Amazon Kinesis
Amazon Simple Queue
Service

Candidate A_2
Candidate B_1
Candidate B_2
Candidate B_3
Candidate B_5
Candidate B_4
Candidate B_7
Candidate B_6
Candidate A_1
Candidate A_3
Candidate A_4
Candidate A_7 Candidate B_8
Candidate A_6 Candidate A_8
Candidate A_5

Candidate A_2
Candidate B_1
Candidate B_2
Candidate B_3
Candidate B_5
Candidate B_4
Candidate B_7
Candidate B_6
Candidate A_1
Candidate A_3
Candidate A_4
Insert: “CandidateA_” + rand(0, 10)
Candidate A_5

Candidate A_2
Candidate B_1
Candidate B_2
Candidate B_3
Candidate B_5
Candidate B_4
Candidate B_7
Candidate B_6
Candidate A_1
Candidate A_3
Candidate A_4
Candidate A_5
Periodic
process
Candidate A
Total: 2.5M
1. Sum
2. Store

Ranking System

• d B DA b
W A
• W e S A
• L T T A a
3
b A
W
e S A

n
D XI o
kRs
BB D d
G t
G e a G A
I yDm G +
r e DS
R
D
Di
UserID(PK) GameID(SK) Score EventID
Alice 1 1000 a01
Bob 1 2000 b01
Bob 2 1000 b02
Charlie 1 3000 c01
Dan 1 500 d01
GameID(P
K)
Score(SK) UserID EvenetID
1 1000 Alice a01
1 1000 Dan d01
1 2000 Bob b01
1 3000 Charlie c01
2 1000 Bob b01
Table
GSI

Amazon Kinesis
Data Firehose
Amazon Athena
Amazon DynamoDB
Amazon ElastiCache
AWS Lambda
Amazon Simple Storage
Service (S3)
Amazon EC2
Amazon Kinesis
Data Streams
•
• ElastiCache DynamoDB Athena
• Elastisearch
DynamoDB
Streams
Redis kinesis
DynamoDB

DynamoDB Streams + Lambda
ElastiCache
Kinesis

Athena

• B Wa H D
•
• -
• e

A Real World Example
“Reality is that which, when you stop believing
in it, does not go away.”

DynamoDB
•
•
•

Access Patterns
# USE CASE
1 CompanionMapping getCompanionMappingsByAsin
2 CompanionMapping getCompanionMappingsByEbookAndAudiobookContentId
3 CompanionMapping getCompanionMappingsFromCache
4 CompanionMapping getCompanionMappings
5 CompanionMapping getCompanionMappingsAvailable
6 AcrInfo getACRInfo
7 AcrInfo getACRs
8 AcrInfo getACRInfos
9 AcrInfo getACRInfosbySKU
10 AudioProduct getAudioProductsForACRs
11 AudioProduct getAudioProducts
12 AudioProduct deleteAudioProductsMatchingSkuVersions
13 AudioProduct getChildAudioProductsForSKU
14 Product getProductInfoByAsins
15 Product getParentChildDataByParentAsins
16 AudioFile getAudioFilesForACR
17 AudioFile getAudioFilesForChildACR
18 AudioFile getAudioFilesByParentAsinVersionFormat
19 AudioFile getAudioFiles
20 AudioFile getAudioFilesForChildAsin

Query Conditions
# USE CASE Lookup parameters INDEX Key Conditions Filter Conditions
1 CompanionMapping getCompanionMappingsByAsin audiobookAsin/ebookSku GSI2 GSI-2=ABOOK-ASIN1 None
2 CompanionMapping
getCompanionMappingsByEbookAndA
udiobookContentId
ebookAcr/sku,version,format or
audiobookAcr/asin,version,format
GSI-3 on TargetACR attribute OR
PrimaryKey on Table
GSI-3=MAP-EBOOKACR1 version=v and format=f
3 CompanionMapping getCompanionMappingsFromCache
ebookAcr/sku,version,format or
audiobookAcr/asin,version,format
GSI-3 on TargetACR attribute OR
PrimaryKey on Table
GSI-3=MAP-EBOOKACR1 version=v and format=f
4 CompanionMapping getCompanionMappings
syncfileAcr, ebookAcr?,
audiobookAcr?
GSI1 GSI-1=SyncFileAcr None
5 CompanionMapping getCompanionMappingsAvailable ebookAcr, audiobookAcr Primary Key on Table
Acr=ABOOKACR1 and
TargetACR beginswith "MAP-"
6 AcrInfo getACRInfo acr Primary Key on Table
Acr=ABOOKACR1 and
TargetACR beginswith "ABOOKACR1-
v"
7 AcrInfo getACRs acr / asin,version,format Primary Key on Table Acr=ABOOKACR1 version=v and format=f
8 AcrInfo getACRInfos acr Primary Key on table
Acr=ABOOKACR1 and
TargetACR beginswith "ABOOKACR1"
9 AcrInfo getACRInfosbySKU sku GSI2 GSI-2=ABOOK-SKU1
10 AudioProduct getAudioProductsForACRs acr Primary Key on table
Acr=ABOOKACR1 and TargetACR
beginswith "ABOOKACR1"
11 AudioProduct getAudioProducts sku, version, format GSI2 GSI-2=ABOOK-SKU1 version=v and format=f
12 AudioProduct
deleteAudioProductsMatchingSkuVersi
ons
sku, version GSI2 GSI-2=ABOOK-SKU1 version=v
13 AudioProduct getChildAudioProductsForSKU sku GSI2 GSI-2=ABOOK-SKU1
14 Product getProductInfoByAsins asin GSI1 GSI-1=ABOOK-ASIN1
15 Product getParentChildDataByParentAsins asin GSI1 GSI-1=ABOOK-ASIN1
16 AudioFile getAudioFilesForACR acr Primary Key on table
Acr=ABOOKACR1 and
TargetACR beginswith "ABOOKACR1#"
17 AudioFile getAudioFilesForChildACR acr, parent_asin Primary Key on table Acr=ABOOKACR1 version=v and format=f
18 AudioFile
getAudioFilesByParentAsinVersionForm
at
parent_asin, version, format GSI1 GSI-1=ABOOK-ASIN1 version=v and format=f
19 AudioFile getAudioFiles sku, version, format GSI2 GSI-2=ABOOK-SKU1 version=v and format=f
20 AudioFile getAudioFilesForChildAsin asin, parent_asin, version, format GSI1 GSI-1=ABOOK-ASIN1 version=v and format=f

Primary Table
T
A
B
L
E
Primary Key
Attributes
PK SK (GSI 3)
ABOOKACR1
v0#ABOOKACR1
GSI-1 GSI-2
ABOOK-ASIN1 ABOOK-SKU1
EBOOKACR1
GSI-1 GSI-2
SyncFileAcr ABOOK-ASIN1
ABOOKACR1#TRACK#1
GSI-1 GSI-2
ABOOKACR1#TRACK#2
GSI-1 GSI-2
EBOOKACR1 EBOOKACR1
GSI-1 EBookAsin
EBOOK-SKU1 ASIN
ASIN Amazon Standard Identification
Number Amazon
10

Indexes
G
S
I
1
Partition Key Projected Attributes
ABOOK-ASIN1 ABOOKACR1
ABOOKACR1-v1
ABOOKACR1#TRACK#1
ABOOKACR1#TRACK#2
SyncFileAcr ABOOKACR1 MAP-EBOOKACR1
EBOOK-SKU1 ABOOKACR1 EBOOKACR1
G
S
I
2
Partition Key Projected Attributes
ABOOK-ASIN1 ABOOKACR1 MAP-EBOOKACR1
ABOOK-SKU1 ABOOKACR1
ABOOKACR1-v1
ABOOKACR1#TRACK#1
ABOOKACR1#TRACK#2
G
S
I
3
GSI Partition Key Projected Attributes
V0#ABOOKACR1 ABOOKACR1 ABOOKACR1-v1
EBOOKACR1 ABOOKACR1 MAP-EBOOKACR1
ABOOKACR1#TRACK#1 ABOOKACR1 ABOOKACR1#TRACK#1
ABOOKACR1#TRACK#2 ABOOKACR1 ABOOKACR1#TRACK#2
EBOOKACR1 ABOOKACR1 EBOOKACR1

• ,B . / w xS e_ NXfdgu V q y Q
a v e
• ,B . L B _ gu V q iPlogz e
Ø t_khnmpjb
B B B = = B L B ? ?B E:H
E ?
Ø LBH H B -2B2
Ø B B = ?B: B B H L B :?B ? B EL
B EB DH EL E BE E H B
• x_ cx b S e_S L B _
• v RRersg dV

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Q&A

https://amzn.to/aws_dev

Thank you!

Devsumi2019 dynamodb

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Devsumi2019 dynamodb

Similar to Devsumi2019 dynamodb (20)

More from oranie Narut

More from oranie Narut (11)

Recently uploaded

Recently uploaded (20)

Devsumi2019 dynamodb