DynamoDB Deep Dive

6,028 views
5,849 views

Published on

Slides do Webinar Deep Dive de DynamoDB, o banco de dados NoSQL gerenciado e as a service da Amazon Web Services

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,028
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
57
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

DynamoDB Deep Dive

  1. 1. Deep dive on DynamoDB to create scalable app Eduardo Horai AWS Solutions Architect
  2. 2. 1 What is DynamoDB?
  3. 3. DynamoDB is a managed NoSQL database service. Store and retrieve any amount of data. Serve any level of request traffic.
  4. 4. Without the operational burden.
  5. 5. Consistent, predictable performance. Single digit millisecond latency. Backed on solid-state drives.
  6. 6. Flexible data model. Key/attribute pairs. No schema required. Easy to create. Easy to adjust.
  7. 7. Seamless scalability. No table size limits. Unlimited storage. No downtime.
  8. 8. Durable. Consistent, disk only writes. Replication across data centers and availability zones.
  9. 9. Focus on your app.
  10. 10. Provisioned throughput. Reserve IOPS for reads and writes. Scale up for down at any time.
  11. 11. Pay per capacity unit. Priced per hour of provisioned throughput.
  12. 12. Write throughput. Size of item x writes per second >= 1KB
  13. 13. Consistent writes. Atomic increment and decrement. Optimistic concurrency control: conditional writes.
  14. 14. Transactions. Item level transactions only. Puts, updates and deletes are ACID.
  15. 15. Strong or eventual consistency Read throughput.
  16. 16. Strong or eventual consistency Read throughput. Provisioned units = size of item x reads per second >= 4KB
  17. 17. Strong or eventual consistency Read throughput. Provisioned units = size of item x reads per second 2
  18. 18. Strong or eventual consistency Read throughput. Same latency expectations. Mix and match at ‘read time’.
  19. 19. Provisioned throughput is managed by DynamoDB.
  20. 20. Data is partitioned and managed by DynamoDB.
  21. 21. Partitioning •  DynamoDB automatically partitions data by the hash key –  Hash key spreads data & workload across partitions •  Auto-Partitioning driven by: –  Data set size –  Provisioned Throughput •  Tip: large number of unique hash keys and uniform distribution of workload across hash keys lends well to massive scale!
  22. 22. Indexed data storage. Tiered bandwidth pricing: aws.amazon.com/dynamodb/pricing
  23. 23. Reserved capacity. Up to 53% for 1 year reservation. Up to 76% for 3 year reservation.
  24. 24. Authentication. Session based to minimize latency. Uses the Amazon Security T oken Service. Handled by AWS SDKs. Integrates with IAM.
  25. 25. Monitoring. CloudWatch metrics: latency, consumed read and write throughput, errors and throttling.
  26. 26. Libraries, mappers and mocks. ColdFusion, Django, Erlang, Java, .Net, Node.js, Perl, PHP Python, Ruby , http://j.mp/dynamodb-libs
  27. 27. 2 NoSQL Data Modeling
  28. 28. id = 100 date = 2012-05-16-09-00-10 total = 25.00 id = 101 date = 2012-05-15-15-00-11 total = 35.00 id = 101 date = 2012-05-16-12-00-10 total = 100.00
  29. 29. T able id = 100 date = 2012-05-16-09-00-10 total = 25.00 id = 101 date = 2012-05-15-15-00-11 total = 35.00 id = 101 date = 2012-05-16-12-00-10 total = 100.00
  30. 30. id = 100 date = 2012-05-16-09-00-10 total = 25.00 id = 101 date = 2012-05-15-15-00-11 total = 35.00 id = 101 date = 2012-05-16-12-00-10 total = 100.00 Item
  31. 31. date = 2012-05-16-09-00-10 total = 25.00 id = 101 date = 2012-05-15-15-00-11 total = 35.00 id = 101 date = 2012-05-16-12-00-10 total = 100.00 id = 100 Attribute
  32. 32. Where is the schema? T ables do not require a formal schema. Items are an arbitrarily sized hash.
  33. 33. Indexing. Items are indexed by primary and secondary keys. Primary keys can be composite. Secondary keys are local to the table.
  34. 34. ID Date T otal
  35. 35. Hash key ID Date T otal
  36. 36. Hash key Range key ID Date Composite primary key T otal
  37. 37. Hash key Range key Secondary range key ID Date T otal
  38. 38. Programming DynamoDB. Small but perfectly formed API.
  39. 39. CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem DescribeTable DeleteItem ListTables Query Scan BatchGetItem BatchWriteItem
  40. 40. CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem DescribeTable DeleteItem ListTables Query Scan BatchGetItem BatchWriteItem
  41. 41. CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem DescribeTable DeleteItem ListTables Query Scan BatchGetItem BatchWriteItem
  42. 42. Conditional updates. PutItem, UpdateItem, DeleteItem can take optional conditions for operation. UpdateItem performs atomic increments.
  43. 43. One API call, multiple items BatchGet returns multiple items by key. BatchWrite performs up to 25 put or delete operations. Throughput is measured by IO, not API calls.
  44. 44. CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem DescribeTable DeleteItem ListTables Query Scan BatchGetItem BatchWriteItem
  45. 45. Query vs Scan Query returns items by key. Scan reads the whole table sequentially.
  46. 46. Query patterns Retrieve all items by hash key. Range key conditions: ==, <, >, >=, <=, begins with, between. Counts. T and bottom n values. op Paged responses.
  47. 47. 3 Example
  48. 48.
  49. 49. AmazonDynamoDBClient dynamoDB; = new AmazonDynamoDBClient( new ClasspathPropertiesFileCredentialsProvider()); dynamoDB.setRegion(Region.getRegion(Regions. SA_EAST_1));
  50. 50. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15
  51. 51. CreateTableRequest createPlayersTable = new CreateTableRequest().withTableName("Players") .withKeySchema(new KeySchemaElement().withAttributeName("user_id") .withKeyType(KeyType.HASH)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName("user_id").withAttributeType(ScalarAttributeType.S)) .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(10L) .withWriteCapacityUnits(10L)); dynamoDB.createTable(createPlayersTable);
  52. 52. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 user_id = mza game = angry-birds score = 11,000 user_id = mza game = tetris score = 1,223,000 user_id = werner game = bejewelled score = 55,000 Scores
  53. 53. CreateTableRequest createScoresTable = new CreateTableRequest().withTableName(”Scores") .withKeySchema(new KeySchemaElement().withAttributeName("user_id") .withKeyType(KeyType.HASH)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName("user_id").withAttributeType(ScalarAttributeType.S)) .withKeySchema(new KeySchemaElement().withAttributeName(”game") .withKeyType(KeyType.RANGE)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName(”game").withAttributeType(ScalarAttributeType.S)) .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(100L) .withWriteCapacityUnits(100L));
  54. 54. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 Leader boards Scores user_id = mza game = angry-birds score = 11,000 game = angry-birds score = 11,000 user_id = mza user_id = mza game = tetris score = 1,223,000 game = tetris score = 1,223,000 user_id = mza user_id = werner game = bejewelled score = 55,000 game = tetris score = 9,000,000 user_id = jeffbarr
  55. 55. CreateTableRequest createLeaderBoardsTable = new CreateTableRequest().withTableName(”LeaderBoards") .withKeySchema(new KeySchemaElement().withAttributeName(”game") .withKeyType(KeyType.HASH)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName(”game").withAttributeType(ScalarAttributeType.S)) .withKeySchema(new KeySchemaElement().withAttributeName(”score") .withKeyType(KeyType.RANGE)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName(”score").withAttributeType(ScalarAttributeType.N)) .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(50L) .withWriteCapacityUnits(50L));
  56. 56. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 Query for user Leader boards Scores user_id = mza game = angry-birds score = 11,000 game = angry-birds score = 11,000 user_id = mza user_id = mza game = tetris score = 1,223,000 game = tetris score = 1,223,000 user_id = mza user_id = werner game = bejewelled score = 55,000 game = tetris score = 9,000,000 user_id = jeffbarr
  57. 57. Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put("user_id", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS("mza"))); QueryRequest queryRequest = new QueryRequest() .withTableName("Players") .withKeyConditions(keyConditions); QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }
  58. 58. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 Query for scores by user Leader boards Scores user_id = mza game = angry-birds score = 11,000 game = angry-birds score = 11,000 user_id = mza user_id = mza game = tetris score = 1,223,000 game = tetris score = 1,223,000 user_id = mza user_id = werner game = bejewelled score = 55,000 game = tetris score = 9,000,000 user_id = jeffbarr
  59. 59. Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put("user_id", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS("mza"))); QueryRequest queryRequest = new QueryRequest() .withTableName(”Scores") .withAttributesToGet(”score”, “game”) .withKeyConditions(keyConditions); QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }
  60. 60. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 Query for scores by user, game Leader boards Scores user_id = mza game = angry-birds score = 11,000 game = angry-birds score = 11,000 user_id = mza user_id = mza game = tetris score = 1,223,000 game = tetris score = 1,223,000 user_id = mza user_id = werner game = bejewelled score = 55,000 game = tetris score = 9,000,000 user_id = jeffbarr
  61. 61. Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put("user_id", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS("mza"))); keyConditions.put(”game", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS(”tetris"))); QueryRequest queryRequest = new QueryRequest() .withTableName(”Scores") .withKeyConditions(keyConditions); QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }
  62. 62. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 High scores by game Leader boards Scores user_id = mza game = angry-birds score = 11,000 game = angry-birds score = 11,000 user_id = mza user_id = mza game = tetris score = 1,223,000 game = tetris score = 1,223,000 user_id = mza user_id = werner game = bejewelled score = 55,000 game = tetris score = 9,000,000 user_id = jeffbarr
  63. 63. Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put(”game", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS(”tetris"))); QueryRequest queryRequest = new QueryRequest() .withTableName(”LeaderBoards") .withKeyConditions(keyConditions) . withScanIndexForward(false); QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }
  64. 64. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 Insert Players Leader boards Scores user_id = mza game = angry-birds score = 11,000 game = angry-birds score = 11,000 user_id = mza user_id = mza game = tetris score = 1,223,000 game = tetris score = 1,223,000 user_id = mza user_id = werner game = bejewelled score = 55,000 game = tetris score = 9,000,000 user_id = jeffbarr
  65. 65. Map<String, AttributeValue> itemPlayer = new HashMap<String, AttributeValue>(); itemPlayer.put("user_id", new AttributeValue("eduardohorai")); itemPlayer.put("location", new AttributeValue("Sao Paulo")); itemPlayer.put("joined", new AttributeValue("27/01/2013")); PutItemRequest putItemRequest = new PutItemRequest("Players", itemPlayer); PutItemResult putItemResult = dynamoDB.putItem(putItemRequest);
  66. 66. Players user_id = mza location = Cambridge joined = 2011-07-04 user_id = jeffbarr location = Seattle joined = 2012-01-20 user_id = werner location = Worldwide joined = 2011-05-15 Increase writes/reads on Scores!!!!! Leader boards Scores user_id = mza game = angry-birds score = 11,000 game = angry-birds score = 11,000 user_id = mza user_id = mza game = tetris score = 1,223,000 game = tetris score = 1,223,000 user_id = mza user_id = werner game = bejewelled score = 55,000 game = tetris score = 9,000,000 user_id = jeffbarr
  67. 67. UpdateTableRequest updateTableRequest = new UpdateTableRequest() .withTableName("Scores") .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(200L) .withWriteCapacityUnits(200L)); UpdateTableResult result = dynamoDB.updateTable(updateTableRequest);
  68. 68. 4 Using AWS Console
  69. 69. Links §  aws.amazon.com/dynamodb   §  aws.typepad.com/brasil/   §  aws.typepad.com   §  awshub.com.br   §  ehorai@amazon.com  
  70. 70. Questions? Learn More: aws.amazon.com/dynamodb
  71. 71. Obrigado! Learn More: aws.amazon.com/dynamodb

×