Dynamo DB & RDS Deep Dive - AWS India Summit 2012


Published on

Dynamo DB & RDS Deep Dive from the AWS Summit in India. Sundar Raghavan presenting.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • If we didn’t do anything we will waste about three-quarters of it’s available capacity. This is the same case for traffic to the database as well where if you don’t have to provision for peak you can save immensely…
  • and DynamoDB enables you to do just that!Even with changes once a week, you can save up to 60%
  • So, who is using DynamoDB?Many companies within Amazon and outside Amazon.Amazon CloudDrive, Smugmug, Amazon retail are examples within Amazon.Smugmug is using us for storing their metadata on DynamoDB.Formpsring and Tapjoy are one of our early adopters.Finally, the shazam the popular voice recognition application for ads and music is using us to store its data for various applications.
  • Dynamo DB & RDS Deep Dive - AWS India Summit 2012

    1. 1. Amazon DynamoDB andAmazon RDS Deep Dive Sundar Raghavan raghavas@amazon
    2. 2. Provides real-time weather updatesMakes video gamers happyScales to millions of usersDelivers billions of page flips How?
    3. 3. Provides real-time weather updatesMakes video gamers happy Amazon DynamoDBScales to millions of users Amazon RDSDelivers billions of page flips
    4. 4. YesSQL? NoSQL? (Hint) Focus on your App
    5. 5. Factors YesSQL (RDS) NoSQL (DynamoDB)Application • App with complex business logic? • Web app with lots of users?Transactions • Complex txns, joins, updates? • Simple data model, updates, queries?Scale • Developer managed • Automatic, on-demand scalingPerformance • Developer architected • Consistent, high performance at scaleAvailability • Architected for fail-over • Seamless and transparentSkills • Relational + Web expertise • Web expertise Best of both worlds: Use SQL and NoSQL models in one app
    6. 6. Focus on your App with Amazon DynamoDB
    7. 7. Amazon DynamoDB
    8. 8. DynamoDB is a managedNoSQL database service.Store and retrieve any amount of data.Serve any level of request traffic.Without any operational burden.
    9. 9. Consistent, predictableperformance.Single digit millisecond latencies.Backed on solid-state drives.
    10. 10. Flexible data modelKey/attribute pairs.No schema required.Easy to create. Easy to adjust.
    11. 11. Seamless scalabilityNo table size limits. Unlimited storage.No downtime.
    12. 12. DurableConsistent, disk-only writes (not memory)Replication across data centres andavailability zones.
    13. 13. Without theoperational burden.
    14. 14. November traffic at Amazon.com 76% 24%
    15. 15. Capacity we needed before DynamoDB Capacity we can provision with DynamoDBActual traffic
    16. 16. Two decisions + threeclicks = ready for use
    17. 17. Two decisions + threeclicks = ready for use
    18. 18. Order Table (collection of items) Attribute (key value pair) String – unicode, utf8 binaryItem (max size 64k, collection of Number – 38 digit precisionattributes) date = 2012-05-16- id = 100 09-00-10 total = 25.00 date = 2012-05-16- id = 101 09-00-11 total = 50.00 Hash Key Range Key
    19. 19. Items are indexed byprimary keySingle hash keys and composite keys.
    20. 20. Think of database operationsas reads and writesItem read = 1 readItem update = 2 reads and 1 writeItem insert = 1 write
    21. 21. Provisioned throughputReserve IOPS for reads and writes.Scale up (or down) at any time.Priced per hour of provisioned throughput.
    22. 22. Provisioned throughputWrite throughput units = item size * writes/sec$0.01 per hour for 10 write unitsRead throughput units = item size * ½ * reads/sec$0.01 per hour for 100 read units
    23. 23. Consistent writesAtomic increment/decrement.Optimistic concurrency control.aka: “conditional writes”.
    24. 24. TransactionsItem level transactions only.Puts, updates and deletes are ACID.
    25. 25. Read throughputProvisioned units =size of item x reads/second$0.01 per hour for 50 read units
    26. 26. Read throughputSame latency expectations.Mix and match at “read time”.
    27. 27. Two decisions + threeclicks = ready for use
    28. 28. Two decisions + threeclicks = ready for use
    29. 29. Two decisions + one API call= ready for use
    30. 30. $create_response = $dynamodb->create_table(array( TableName => ProductCatalog, KeySchema => array( HashKeyElement => array( AttributeName => Id, AttributeType => AmazonDynamoDB::TYPE_NUMBER ) ), ProvisionedThroughput => array( ReadCapacityUnits => 10, WriteCapacityUnits => 5 )));
    31. 31. Two decisions + one API call= ready for use
    32. 32. Two decisions + one API call= ready for development
    33. 33. Two decisions + one API call= ready for production
    34. 34. Two decisions + one API call= ready for scale
    35. 35. AuthenticationSession based to minimize latency.Uses Amazon Security Token Service.Handled by AWS SDKs.Integrates with IAM.
    36. 36. MonitoringCloudWatch metrics:latency, consumed read and writethroughput, errors and throttling.
    37. 37. Libraries, mappers &mocksColdFusion, Django, Erlang, Java, .Net,Node.js, Perl, PHP, Python, Rubyhttp://j.mp/dynamodb-libs
    38. 38. (And I can‟t find my joins)Ma, I lost my schema!Tables do not require a formal schema.Items are an arbitrary sized hash.Just need to specify the primary key.
    39. 39. Programming DynamoDBSmall but perfectly formed.Whole programming interfacefits on one slide.
    40. 40. CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem “Select”, “insert”, DescribeTable “update” itemsManage tables DeleteItem ListTables BatchGetItem Query Bulk select or Query specific Scan BatchWriteItem update (max 1MB) items OR scan the full table
    41. 41. Query patternsRetrieve all items by hash key.Range key conditions:==, <, >, >=, <=, begins with, between.Counts. Top and bottom n values.Paged responses.
    42. 42. Modeling patterns
    43. 43. 1. Mapping relationshipswith range keys.No cross-table joins in DynamoDB.Use composite keys to modelrelationships.
    44. 44. 2. Handling large items.Unlimited attributes per item.Unlimited items per table.Max 64k per item.
    45. 45. Store a pointer to objects inAmazon S3Large data stored in S3.Location stored in DynamoDB.99.999999999% data durability in S3.
    46. 46. 3. Managing secondaryindices.Not currently supported by DynamoDB.You can create on your own.
    47. 47. 4. Time series data.Logging, click through, ad views,game play data, application usage.Non-uniform access patterns.Newer data is „live‟.Older data is read only.
    48. 48. Hot and cold tables.Dec Jan Feb Mar April May lower higher throughput throughput
    49. 49. Amazon CloudDrive• Stores user objects in Cloud – Metadata in DynamoDB – Objects in S3• Queries and object searches are served by DynamoDB – What are all the objects in my drive? – Find me all the music albums stored on my drive “We were excited by how fast we were able to put DynamoDB into production and how much developer time we saved. In addition, DynamoDB lets us scale up and down easily by simply reserving increased throughput capacity when we need it and dialing it back when we don’t” - Russel Dicker, Amazon CloudDrive
    50. 50. DynamoDB Customer Highlights
    51. 51. Focus on your App with Amazon RDS
    52. 52. Load balancerApplication tierDatabase tier
    53. 53. Load balancer Application tier Invest more time making this innovative Database tierSpend less time managing this
    54. 54. Migration Backup andSchema design recovery Patching Query constructionConfiguration Software upgrades Frequent server Storage upgrades upgrades Hardware crash Query optimization
    55. 55. Migration Backup and recoverySchema design PatchingQuery construction ConfigurationQuery optimization Software upgrades Storage upgrades Focus on the “buck” Frequent server upgrades Hardware crash Off load the “muck”
    56. 56. Security planning License training Backup, recovery load and unload Script automation Performance Installation, upgrade, and tuning patching, migrationSource: Forrester
    57. 57. Amazon RDSRelational Database Service Quick introduction
    58. 58. Amazon RDS OverviewAmazon RDS is a fully managed relationaldatabase service. Choice of Database engines Simple to deploy Easy to operate and scale Reliable Cost effective
    59. 59. Choice of database engines MySQL Oracle Microsoft SQL Server
    60. 60. Three decisions + a few clicks= ready for use
    61. 61. Easy to changeThree decisions + a few clicks= ready for use
    62. 62. Easy to changeThree decisions + a few clicks= ready for use
    63. 63. Three decisions + a few clicks= ready for use
    64. 64. Three decisions + a few clicks= ready for development
    65. 65. Three decisions + a few clicks= ready for scale
    66. 66. Three decisions + a few clicks= ready for production
    67. 67. Three ways to remove the “muck”
    68. 68. Productivity PerformanceData Protection
    69. 69. ProductivityMost admin tasks are managed by RDS Provisioning Database backups Patching Performance management
    70. 70. ProductivityDatabase snapshots Automatic Point in time recovery Faster time to recover
    71. 71. Productivity Multiple databases per instance Standard user accounts Connect and query using common MySQL tools & drivers Tune engine parameters Import and export data using standard MySQL tools (mysqldump) Diagnostics Native MySQL replication SSL for encryption over the wire Monitor metrics Shell, super user or direct file system access (Think security!)
    72. 72. ProductivityNear zero administration Painless patching, automatic upgrades Cloudwatch monitoring, Metric alarms One click. High Availability.
    73. 73. ProductivityOne click. High availability with Multi-AZ Automated deployment across multiple AZs Synchronous replication from master to replica Automatic fail-over; replica promoted to master Test fail-over
    74. 74. PerformancePush-button scale, high performance Scale storage from 5Gb to 1Tb of storage Scale instance from small to 4XL (better I/O) Add read replicas with asynchronous replication Add ElastiCache for performance
    75. 75. PerformanceAmazon ElastiCache In-memory cache service Ideal front end to RDS for read-heavy applications Low latency
    76. 76. Data ProtectionPainless data protection Daily database back-ups (save up to 35 days) Snapshot any time. Point in time restore-use innodb Recover everything up to final 5 minutes-innodb! VPC, SSL, DB Security groups
    77. 77. Samsung saved $34M Problem: • Needed to reduce IT costs and were looking to create a more flexible IT environment Solution: • AWS and RDS services. With every request, the application authenticates devices, delivers apps and content, and pushes notifications. Business Benefits: • Saved $34M in hardware and maintenance expenses, 85% less than running on-premises
    78. 78. YesSQL NoSQL Focus on your App
    79. 79. Amazon RDS Customer HighlightsGaming Web Apps Mobile/Social Media
    80. 80. Thank youFree tier or Free trial aws.amazon.com/dynamodb aws.amazon.com/rds raghavas@amazon