Slides do Webinar Deep Dive de DynamoDB, o banco de dados NoSQL gerenciado e as a service da Amazon Web Services

1. Deep dive on DynamoDB to create scalable app
Eduardo Horai
AWS Solutions Architect

2. 1
What is
DynamoDB?

3. DynamoDB is a managed
NoSQL database service.
Store and retrieve any amount of data.
Serve any level of request traffic.

4. Without the operational burden.

5. Consistent, predictable performance.
Single digit millisecond latency.
Backed on solid-state drives.

6. Flexible data model.
Key/attribute pairs. No schema required.
Easy to create. Easy to adjust.

7. Seamless scalability.
No table size limits. Unlimited storage.
No downtime.

8. Durable.
Consistent, disk only writes.
Replication across data centers and availability zones.

9. Focus on your app.

10. Provisioned throughput.
Reserve IOPS for reads and writes.
Scale up for down at any time.

11. Pay per capacity unit.
Priced per hour of provisioned throughput.

12. Write throughput.
Size of item x writes per second
>= 1KB

13. Consistent writes.
Atomic increment and decrement.
Optimistic concurrency control: conditional writes.

14. Transactions.
Item level transactions only.
Puts, updates and deletes are ACID.

15. Strong or eventual consistency
Read throughput.

16. Strong or eventual consistency
Read throughput.
Provisioned units = size of item x reads per second
>= 4KB

17. Strong or eventual consistency
Read throughput.
Provisioned units = size of item x reads per second
2

18. Strong or eventual consistency
Read throughput.
Same latency expectations.
Mix and match at ‘read time’.

19. Provisioned throughput is
managed by DynamoDB.

20. Data is partitioned and
managed by DynamoDB.

21. Partitioning
• DynamoDB automatically partitions data by the hash key
– Hash key spreads data & workload across partitions
• Auto-Partitioning driven by:
– Data set size
– Provisioned Throughput
• Tip: large number of unique hash keys and uniform
distribution of workload across hash keys lends well to
massive scale!

22. Indexed data storage.
Tiered bandwidth pricing:
aws.amazon.com/dynamodb/pricing

23. Reserved capacity.
Up to 53% for 1 year reservation.
Up to 76% for 3 year reservation.

24. Authentication.
Session based to minimize latency.
Uses the Amazon Security T
oken Service.
Handled by AWS SDKs.
Integrates with IAM.

25. Monitoring.
CloudWatch metrics:
latency, consumed read and write throughput,
errors and throttling.

26. Libraries, mappers and mocks.
ColdFusion, Django, Erlang, Java, .Net,
Node.js, Perl, PHP Python, Ruby
,
http://j.mp/dynamodb-libs

27. 2
NoSQL Data
Modeling

28. id = 100
date =
2012-05-16-09-00-10
total = 25.00
id = 101
date =
2012-05-15-15-00-11
total = 35.00
id = 101
date =
2012-05-16-12-00-10
total = 100.00

29. T
able
id = 100
date =
2012-05-16-09-00-10
total = 25.00
id = 101
date =
2012-05-15-15-00-11
total = 35.00
id = 101
date =
2012-05-16-12-00-10
total = 100.00

30. id = 100
date =
2012-05-16-09-00-10
total = 25.00
id = 101
date =
2012-05-15-15-00-11
total = 35.00
id = 101
date =
2012-05-16-12-00-10
total = 100.00
Item

31. date =
2012-05-16-09-00-10
total = 25.00
id = 101
date =
2012-05-15-15-00-11
total = 35.00
id = 101
date =
2012-05-16-12-00-10
total = 100.00
id = 100
Attribute

32. Where is the schema?
T
ables do not require a formal schema.
Items are an arbitrarily sized hash.

33. Indexing.
Items are indexed by primary and secondary keys.
Primary keys can be composite.
Secondary keys are local to the table.

34. ID
Date
T
otal

35. Hash key
ID
Date
T
otal

36. Hash key
Range key
ID
Date
Composite primary key
T
otal

37. Hash key
Range key
Secondary range key
ID
Date
T
otal

38. Programming DynamoDB.
Small but perfectly formed API.

39. CreateTable
PutItem
UpdateTable
GetItem
DeleteTable
UpdateItem
DescribeTable
DeleteItem
ListTables
Query
Scan
BatchGetItem
BatchWriteItem

40. CreateTable
PutItem
UpdateTable
GetItem
DeleteTable
UpdateItem
DescribeTable
DeleteItem
ListTables
Query
Scan
BatchGetItem
BatchWriteItem

41. CreateTable
PutItem
UpdateTable
GetItem
DeleteTable
UpdateItem
DescribeTable
DeleteItem
ListTables
Query
Scan
BatchGetItem
BatchWriteItem

42. Conditional updates.
PutItem, UpdateItem, DeleteItem can take
optional conditions for operation.
UpdateItem performs atomic increments.

43. One API call, multiple items
BatchGet returns multiple items by key.
BatchWrite performs up to 25 put or delete operations.
Throughput is measured by IO, not API calls.

44. CreateTable
PutItem
UpdateTable
GetItem
DeleteTable
UpdateItem
DescribeTable
DeleteItem
ListTables
Query
Scan
BatchGetItem
BatchWriteItem

45. Query vs Scan
Query returns items by key.
Scan reads the whole table sequentially.

46. Query patterns
Retrieve all items by hash key.
Range key conditions:
==, <, >, >=, <=, begins with, between.
Counts. T and bottom n values.
op
Paged responses.

47. 3
Example

48. …

49. AmazonDynamoDBClient dynamoDB; = new AmazonDynamoDBClient(
new ClasspathPropertiesFileCredentialsProvider());
dynamoDB.setRegion(Region.getRegion(Regions. SA_EAST_1));

50. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15

51. CreateTableRequest createPlayersTable =
new CreateTableRequest().withTableName("Players")
.withKeySchema(new KeySchemaElement().withAttributeName("user_id")
.withKeyType(KeyType.HASH))
.withAttributeDefinitions(newAttributeDefinition()
.withAttributeName("user_id").withAttributeType(ScalarAttributeType.S))
.withProvisionedThroughput(new ProvisionedThroughput()
.withReadCapacityUnits(10L)
.withWriteCapacityUnits(10L));
dynamoDB.createTable(createPlayersTable);

52. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
user_id =
mza
game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
werner
game =
bejewelled
score =
55,000
Scores

53. CreateTableRequest createScoresTable =
new CreateTableRequest().withTableName(”Scores")
.withKeySchema(new KeySchemaElement().withAttributeName("user_id")
.withKeyType(KeyType.HASH))
.withAttributeDefinitions(newAttributeDefinition()
.withAttributeName("user_id").withAttributeType(ScalarAttributeType.S))
.withKeySchema(new KeySchemaElement().withAttributeName(”game")
.withKeyType(KeyType.RANGE))
.withAttributeDefinitions(newAttributeDefinition()
.withAttributeName(”game").withAttributeType(ScalarAttributeType.S))
.withProvisionedThroughput(new ProvisionedThroughput()
.withReadCapacityUnits(100L)
.withWriteCapacityUnits(100L));

54. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Leader boards
Scores
user_id =
mza
game =
angry-birds
score =
11,000
game =
angry-birds
score =
11,000
user_id =
mza
user_id =
mza
game =
tetris
score =
1,223,000
game =
tetris
score =
1,223,000
user_id =
mza
user_id =
werner
game =
bejewelled
score =
55,000
game =
tetris
score =
9,000,000
user_id =
jeffbarr

55. CreateTableRequest createLeaderBoardsTable =
new CreateTableRequest().withTableName(”LeaderBoards")
.withKeySchema(new KeySchemaElement().withAttributeName(”game")
.withKeyType(KeyType.HASH))
.withAttributeDefinitions(newAttributeDefinition()
.withAttributeName(”game").withAttributeType(ScalarAttributeType.S))
.withKeySchema(new KeySchemaElement().withAttributeName(”score")
.withKeyType(KeyType.RANGE))
.withAttributeDefinitions(newAttributeDefinition()
.withAttributeName(”score").withAttributeType(ScalarAttributeType.N))
.withProvisionedThroughput(new ProvisionedThroughput()
.withReadCapacityUnits(50L)
.withWriteCapacityUnits(50L));

56. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Query for user
Leader boards
Scores
user_id =
mza
game =
angry-birds
score =
11,000
game =
angry-birds
score =
11,000
user_id =
mza
user_id =
mza
game =
tetris
score =
1,223,000
game =
tetris
score =
1,223,000
user_id =
mza
user_id =
werner
game =
bejewelled
score =
55,000
game =
tetris
score =
9,000,000
user_id =
jeffbarr

57. Map<String, Condition> keyConditions = new HashMap<String, Condition>();
keyConditions.put("user_id", new Condition()
.withComparisonOperator(ComparisonOperator.EQ.toString())
.withAttributeValueList(new AttributeValue().withS("mza")));
QueryRequest queryRequest = new QueryRequest()
.withTableName("Players")
.withKeyConditions(keyConditions);
QueryResult result = dynamoDB.query(queryRequest);
for (Map<String, AttributeValue> item : result.getItems()) {
printItem(item);
}

58. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Query for scores
by user
Leader boards
Scores
user_id =
mza
game =
angry-birds
score =
11,000
game =
angry-birds
score =
11,000
user_id =
mza
user_id =
mza
game =
tetris
score =
1,223,000
game =
tetris
score =
1,223,000
user_id =
mza
user_id =
werner
game =
bejewelled
score =
55,000
game =
tetris
score =
9,000,000
user_id =
jeffbarr

59. Map<String, Condition> keyConditions = new HashMap<String, Condition>();
keyConditions.put("user_id", new Condition()
.withComparisonOperator(ComparisonOperator.EQ.toString())
.withAttributeValueList(new AttributeValue().withS("mza")));
QueryRequest queryRequest = new QueryRequest()
.withTableName(”Scores")
.withAttributesToGet(”score”, “game”)
.withKeyConditions(keyConditions);
QueryResult result = dynamoDB.query(queryRequest);
for (Map<String, AttributeValue> item : result.getItems()) {
printItem(item);
}

60. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Query for scores
by user, game
Leader boards
Scores
user_id =
mza
game =
angry-birds
score =
11,000
game =
angry-birds
score =
11,000
user_id =
mza
user_id =
mza
game =
tetris
score =
1,223,000
game =
tetris
score =
1,223,000
user_id =
mza
user_id =
werner
game =
bejewelled
score =
55,000
game =
tetris
score =
9,000,000
user_id =
jeffbarr

61. Map<String, Condition> keyConditions = new HashMap<String, Condition>();
keyConditions.put("user_id", new Condition()
.withComparisonOperator(ComparisonOperator.EQ.toString())
.withAttributeValueList(new AttributeValue().withS("mza")));
keyConditions.put(”game", new Condition()
.withComparisonOperator(ComparisonOperator.EQ.toString())
.withAttributeValueList(new AttributeValue().withS(”tetris")));
QueryRequest queryRequest = new QueryRequest()
.withTableName(”Scores")
.withKeyConditions(keyConditions);
QueryResult result = dynamoDB.query(queryRequest);
for (Map<String, AttributeValue> item : result.getItems()) {
printItem(item);
}

62. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
High scores by game
Leader boards
Scores
user_id =
mza
game =
angry-birds
score =
11,000
game =
angry-birds
score =
11,000
user_id =
mza
user_id =
mza
game =
tetris
score =
1,223,000
game =
tetris
score =
1,223,000
user_id =
mza
user_id =
werner
game =
bejewelled
score =
55,000
game =
tetris
score =
9,000,000
user_id =
jeffbarr

63. Map<String, Condition> keyConditions = new HashMap<String, Condition>();
keyConditions.put(”game", new Condition()
.withComparisonOperator(ComparisonOperator.EQ.toString())
.withAttributeValueList(new AttributeValue().withS(”tetris")));
QueryRequest queryRequest = new QueryRequest()
.withTableName(”LeaderBoards")
.withKeyConditions(keyConditions)
. withScanIndexForward(false);
QueryResult result = dynamoDB.query(queryRequest);
for (Map<String, AttributeValue> item : result.getItems()) {
printItem(item);
}

64. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Insert Players
Leader boards
Scores
user_id =
mza
game =
angry-birds
score =
11,000
game =
angry-birds
score =
11,000
user_id =
mza
user_id =
mza
game =
tetris
score =
1,223,000
game =
tetris
score =
1,223,000
user_id =
mza
user_id =
werner
game =
bejewelled
score =
55,000
game =
tetris
score =
9,000,000
user_id =
jeffbarr

65. Map<String, AttributeValue> itemPlayer =
new HashMap<String, AttributeValue>();
itemPlayer.put("user_id", new AttributeValue("eduardohorai"));
itemPlayer.put("location", new AttributeValue("Sao Paulo"));
itemPlayer.put("joined", new AttributeValue("27/01/2013"));
PutItemRequest putItemRequest =
new PutItemRequest("Players", itemPlayer);
PutItemResult putItemResult = dynamoDB.putItem(putItemRequest);

66. Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Increase writes/reads
on Scores!!!!!
Leader boards
Scores
user_id =
mza
game =
angry-birds
score =
11,000
game =
angry-birds
score =
11,000
user_id =
mza
user_id =
mza
game =
tetris
score =
1,223,000
game =
tetris
score =
1,223,000
user_id =
mza
user_id =
werner
game =
bejewelled
score =
55,000
game =
tetris
score =
9,000,000
user_id =
jeffbarr

67. UpdateTableRequest updateTableRequest = new UpdateTableRequest()
.withTableName("Scores")
.withProvisionedThroughput(new ProvisionedThroughput()
.withReadCapacityUnits(200L)
.withWriteCapacityUnits(200L));
UpdateTableResult result =
dynamoDB.updateTable(updateTableRequest);

68. 4
Using
AWS Console

69. Links
§ aws.amazon.com/dynamodb
§ aws.typepad.com/brasil/
§ aws.typepad.com
§ awshub.com.br
§ ehorai@amazon.com

70. Questions?
Learn More:
aws.amazon.com/dynamodb

71. Obrigado!
Learn More:
aws.amazon.com/dynamodb