Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Re:Invent 2018 Database Announcements

65 views

Published on

A lightning talk about the new AWS databases: Timestream, MSK, QLDB, and Lake Formation.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Re:Invent 2018 Database Announcements

  1. 1. steven@claritycloudworks.com +64 27 620 1237 claritycloudworks.com Steven Ensslen
  2. 2. DynamoDB Transactions data = await dynamoDb.transactWriteItems({ TransactItems: [ { Update: { TableName: 'items', Key: { id: { S: itemId } }, ConditionExpression: 'available = :true', UpdateExpression: 'set available = :false, ' + 'ownedBy = :player', ExpressionAttributeValues: { ':true': { BOOL: true }, ':false': { BOOL: false }, ':player': { S: playerId } } } }, { Update: { TableName: 'players', Key: { id: { S: playerId } }, ConditionExpression: 'coins >= :price', UpdateExpression: 'set coins = coins - :price, ' + 'inventory = list_append(inventory, :items)', ExpressionAttributeValues: { ':items': { L: [{ S: itemId }] }, ':price': { N: itemPrice.toString() } } } } ] }).promise(); https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transactions/ Makes Dynamo “ACID” 
 => suitable for more complex workloads
  3. 3. DynamoDB On-Demand https://aws.amazon.com/blogs/database/amazon-dynamodb-on-demand-no- capacity-planning-and-pay-per-request-pricing/ DynamoDB now has two pricing models: 1. Provisioned, which is cheaper if your load is readily forecasted and enough load to cover the cost making the forecast. 
 $0.00074/WCU / 3600 seconds = $0.205 per million writes 2. On-demand, which is more expensive per request, but saves the effort of capacity planning and the cost of excess capacity. $1.423 per million Writes
  4. 4. Aurora Global DB https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora- global-database.html A single Aurora database can now be multi-regional. • One region is read/write. • One region is read-only. • Today, failover is entirely your responsibility, including preventing split-brain.
  5. 5. RDS on VMWare(preview) https://aws.amazon.com/rds/vmware/ You provide the hardware, AWS provides the software 1. Outsource Database Administration to AWS 2. Burst capacity (scale out) into AWS 3. High Availability and Disaster Recovery via AWS 4. Cloud Migration 
 (alternative to Database Migration Service)
  6. 6. Managed Service for Kafka (MSK) (public preview) https://aws.amazon.com/msk/ Feature set very similar to Kinesis - direct competitor MSK use cases: • Existing Kafka application code • Exactly once delivery Pros: • Cheaper if your server is at capacity Cons: • Unit of scaling is more coarse (1 M5.large vs 1 MB/s) • Scaling is not seamless to application.
  7. 7. Timestream(preview) https://aws.amazon.com/timestream/ • Timeseries Database • 1000 times faster than RDS for time series data • Time series specific compression and query engine. • Built-in interpolation and smoothing • Serverless
  8. 8. Quantum Ledger Database (preview) https://aws.amazon.com/qldb/ • Blockchain as a service • No relation to quantum computing • Immutable and Persistent journal ~= Transaction Log • Reliable and irrefutable “Created By” and “Updated By” • Document data model with SQL • Twice as fast as blockchain => 500 times slower than RDS? • AWS has been using this internally for billing for “years” • Finally a means to implement the intent of the US Sarbanes-Oxley Act of 2002
  9. 9. Lake formation (preview) https://aws.amazon.com/lake-formation/ • No cost, just co-ordinates other services • mostly generates AWS Glue (pyspark) modules • Undifferentiated heavy data lake lifting: • Ingest • Tranform to ORC or Parquet • Encrypt • Secure
  10. 10. Clarity Cloudworks illuminating issues before they become problems

×