Re:Invent 2018 Database Announcements

Steven Ensslen
Steven EnsslenCloud Data & Business Intelligence Architect
steven@claritycloudworks.com
+64 27 620 1237
claritycloudworks.com
Steven Ensslen
DynamoDB Transactions
data = await dynamoDb.transactWriteItems({
TransactItems: [
{ Update: {
TableName: 'items',
Key: { id: { S: itemId } },
ConditionExpression: 'available = :true',
UpdateExpression: 'set available = :false, ' +
'ownedBy = :player',
ExpressionAttributeValues: {
':true': { BOOL: true },
':false': { BOOL: false },
':player': { S: playerId }
} } },
{ Update: {
TableName: 'players',
Key: { id: { S: playerId } },
ConditionExpression: 'coins >= :price',
UpdateExpression: 'set coins = coins - :price, ' +
'inventory = list_append(inventory, :items)',
ExpressionAttributeValues: {
':items': { L: [{ S: itemId }] },
':price': { N: itemPrice.toString() }
} } }
]
}).promise();
https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transactions/
Makes Dynamo “ACID” 

=> suitable for more complex
workloads
DynamoDB On-Demand
https://aws.amazon.com/blogs/database/amazon-dynamodb-on-demand-no-
capacity-planning-and-pay-per-request-pricing/
DynamoDB now has two pricing models:
1. Provisioned, which is cheaper if your load is readily forecasted
and enough load to cover the cost making the forecast. 

$0.00074/WCU / 3600 seconds = $0.205 per million writes
2. On-demand, which is more expensive per request, but saves
the effort of capacity planning and the cost of excess
capacity. $1.423 per million Writes
Aurora Global DB
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-
global-database.html
A single Aurora database can now be multi-regional.
• One region is read/write.
• One region is read-only.
• Today, failover is entirely your responsibility, including
preventing split-brain.
RDS on VMWare(preview)
https://aws.amazon.com/rds/vmware/
You provide the hardware, AWS provides the software
1. Outsource Database Administration to AWS
2. Burst capacity (scale out) into AWS
3. High Availability and Disaster Recovery via AWS
4. Cloud Migration 

(alternative to Database Migration Service)
Managed Service for Kafka (MSK) (public preview)
https://aws.amazon.com/msk/
Feature set very similar to Kinesis - direct competitor
MSK use cases:
• Existing Kafka application code
• Exactly once delivery
Pros:
• Cheaper if your server is at capacity
Cons:
• Unit of scaling is more coarse (1 M5.large vs 1 MB/s)
• Scaling is not seamless to application.
Timestream(preview)
https://aws.amazon.com/timestream/
• Timeseries Database
• 1000 times faster than RDS for time series data
• Time series specific compression and query engine.
• Built-in interpolation and smoothing
• Serverless
Quantum Ledger Database (preview)
https://aws.amazon.com/qldb/
• Blockchain as a service
• No relation to quantum computing
• Immutable and Persistent journal ~= Transaction Log
• Reliable and irrefutable “Created By” and “Updated By”
• Document data model with SQL
• Twice as fast as blockchain => 500 times slower than RDS?
• AWS has been using this internally for billing for “years”
• Finally a means to implement the intent of the US Sarbanes-Oxley
Act of 2002
Lake formation (preview)
https://aws.amazon.com/lake-formation/
• No cost, just co-ordinates other services
• mostly generates AWS Glue (pyspark) modules
• Undifferentiated heavy data lake lifting:
• Ingest
• Tranform to ORC or Parquet
• Encrypt
• Secure
Clarity Cloudworks
illuminating issues before they become problems
1 of 10

Recommended

Serverless GraphQL. AppSync 101 by
Serverless GraphQL. AppSync 101Serverless GraphQL. AppSync 101
Serverless GraphQL. AppSync 101Marcin Sodkiewicz
460 views42 slides
AWS user group Serverless in September - Chris Johnson Bidler "Go Serverless ... by
AWS user group Serverless in September - Chris Johnson Bidler "Go Serverless ...AWS user group Serverless in September - Chris Johnson Bidler "Go Serverless ...
AWS user group Serverless in September - Chris Johnson Bidler "Go Serverless ...AWS Chicago
216 views24 slides
How Docker Accelerates Continuous Development at ironSource: Containers #101 ... by
How Docker Accelerates Continuous Development at ironSource: Containers #101 ...How Docker Accelerates Continuous Development at ironSource: Containers #101 ...
How Docker Accelerates Continuous Development at ironSource: Containers #101 ...Brittany Ingram
1.3K views33 slides
Amazon Web Services (cloud: is it good for anything?) by
Amazon Web Services (cloud: is it good for anything?)Amazon Web Services (cloud: is it good for anything?)
Amazon Web Services (cloud: is it good for anything?)Maciej Pasternacki
1.1K views19 slides
XWiki: The best wiki for developers by
XWiki: The best wiki for developersXWiki: The best wiki for developers
XWiki: The best wiki for developersVincent Massol
215 views30 slides
AWS Bulgaria: Re:invent 2017 Recap by
AWS Bulgaria: Re:invent 2017 RecapAWS Bulgaria: Re:invent 2017 Recap
AWS Bulgaria: Re:invent 2017 RecapIvaylo Bratoev
79 views29 slides

More Related Content

What's hot

Amazon Web Services lection 5 by
Amazon Web Services lection 5  Amazon Web Services lection 5
Amazon Web Services lection 5 Binary Studio
234 views12 slides
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your Data by
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your DataMongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your Data
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your DataMongoDB
438 views90 slides
PloneConf2017: serverless python for astronaut safety by
PloneConf2017:  serverless python for astronaut safetyPloneConf2017:  serverless python for astronaut safety
PloneConf2017: serverless python for astronaut safetyChris Shenton
214 views9 slides
RedisConf18 - Redis and Elasticsearch by
RedisConf18 - Redis and ElasticsearchRedisConf18 - Redis and Elasticsearch
RedisConf18 - Redis and ElasticsearchRedis Labs
1.3K views56 slides
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ... by
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...MongoDB
478 views32 slides
Elasticsearch @ Keboola by
Elasticsearch @ KeboolaElasticsearch @ Keboola
Elasticsearch @ KeboolaMartin Halamíček
714 views12 slides

What's hot(20)

Amazon Web Services lection 5 by Binary Studio
Amazon Web Services lection 5  Amazon Web Services lection 5
Amazon Web Services lection 5
Binary Studio234 views
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your Data by MongoDB
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your DataMongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your Data
MongoDB .local Toronto 2019: Using Change Streams to Keep Up with Your Data
MongoDB438 views
PloneConf2017: serverless python for astronaut safety by Chris Shenton
PloneConf2017:  serverless python for astronaut safetyPloneConf2017:  serverless python for astronaut safety
PloneConf2017: serverless python for astronaut safety
Chris Shenton214 views
RedisConf18 - Redis and Elasticsearch by Redis Labs
RedisConf18 - Redis and ElasticsearchRedisConf18 - Redis and Elasticsearch
RedisConf18 - Redis and Elasticsearch
Redis Labs1.3K views
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ... by MongoDB
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...
MongoDB .local Paris 2020: Adéo @MongoDB : MongoDB Atlas & Leroy Merlin : et ...
MongoDB478 views
Streaming, Analytics and Reactive Applications with Apache Cassandra by Cédrick Lunven
Streaming, Analytics and Reactive Applications with Apache CassandraStreaming, Analytics and Reactive Applications with Apache Cassandra
Streaming, Analytics and Reactive Applications with Apache Cassandra
Cédrick Lunven106 views
If you give a mouse a clickhouse, by Alex Hofsteede, Sentry by Altinity Ltd
If you give a mouse a clickhouse, by Alex Hofsteede, SentryIf you give a mouse a clickhouse, by Alex Hofsteede, Sentry
If you give a mouse a clickhouse, by Alex Hofsteede, Sentry
Altinity Ltd2.4K views
Amazon Web Services lection 4 by Binary Studio
Amazon Web Services lection 4  Amazon Web Services lection 4
Amazon Web Services lection 4
Binary Studio192 views
Azure management portal by Amal Dev
Azure management portalAzure management portal
Azure management portal
Amal Dev519 views
Build Your Web Analytics with node.js, Amazon DynamoDB and Amazon EMR (BDT203... by Amazon Web Services
Build Your Web Analytics with node.js, Amazon DynamoDB and Amazon EMR (BDT203...Build Your Web Analytics with node.js, Amazon DynamoDB and Amazon EMR (BDT203...
Build Your Web Analytics with node.js, Amazon DynamoDB and Amazon EMR (BDT203...
Javascript Everywhere From Nose To Tail by Cliffano Subagio
Javascript Everywhere From Nose To TailJavascript Everywhere From Nose To Tail
Javascript Everywhere From Nose To Tail
Cliffano Subagio3.8K views
Benchx: An XQuery benchmarking web application by Andy Bunce
Benchx: An XQuery benchmarking web application Benchx: An XQuery benchmarking web application
Benchx: An XQuery benchmarking web application
Andy Bunce1.5K views
Migrating Data Pipeline from MongoDB to Cassandra by Demi Ben-Ari
Migrating Data Pipeline from MongoDB to CassandraMigrating Data Pipeline from MongoDB to Cassandra
Migrating Data Pipeline from MongoDB to Cassandra
Demi Ben-Ari2.3K views
Server-less solution for moving Millions of Images in Cloud - Brett Sutter, ... by AWS Chicago
 Server-less solution for moving Millions of Images in Cloud - Brett Sutter, ... Server-less solution for moving Millions of Images in Cloud - Brett Sutter, ...
Server-less solution for moving Millions of Images in Cloud - Brett Sutter, ...
AWS Chicago61 views
Amazon’s Cloud Computing Efforts by awesomesos
Amazon’s Cloud Computing EffortsAmazon’s Cloud Computing Efforts
Amazon’s Cloud Computing Efforts
awesomesos763 views
Running a DynamoDB-compatible Database on Managed Kubernetes Services by ScyllaDB
Running a DynamoDB-compatible Database on Managed Kubernetes ServicesRunning a DynamoDB-compatible Database on Managed Kubernetes Services
Running a DynamoDB-compatible Database on Managed Kubernetes Services
ScyllaDB881 views
From MySQL to MongoDB at Wordnik (Tony Tam) by MongoSF
From MySQL to MongoDB at Wordnik (Tony Tam)From MySQL to MongoDB at Wordnik (Tony Tam)
From MySQL to MongoDB at Wordnik (Tony Tam)
MongoSF3.5K views

Similar to Re:Invent 2018 Database Announcements

[JCConf 2020] 用 Kotlin 跨入 Serverless 世代 by
[JCConf 2020] 用 Kotlin 跨入 Serverless 世代[JCConf 2020] 用 Kotlin 跨入 Serverless 世代
[JCConf 2020] 用 Kotlin 跨入 Serverless 世代Shengyou Fan
623 views80 slides
AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306) by
AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306)AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306)
AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306)Amazon Web Services
2.2K views86 slides
The fundamentals of AWS Cloud Security 🛠⛅️🚀 by
The fundamentals of AWS Cloud Security 🛠⛅️🚀The fundamentals of AWS Cloud Security 🛠⛅️🚀
The fundamentals of AWS Cloud Security 🛠⛅️🚀Thanh Nguyen
761 views65 slides
Riga dev day: Lambda architecture at AWS by
Riga dev day: Lambda architecture at AWSRiga dev day: Lambda architecture at AWS
Riga dev day: Lambda architecture at AWSAntons Kranga
1.2K views43 slides
Amazon Web Services for PHP Developers by
Amazon Web Services for PHP DevelopersAmazon Web Services for PHP Developers
Amazon Web Services for PHP DevelopersJeremy Lindblom
3K views53 slides
What Is Happening At The Edge by
What Is Happening At The EdgeWhat Is Happening At The Edge
What Is Happening At The EdgeAmazon Web Services
433 views50 slides

Similar to Re:Invent 2018 Database Announcements(20)

[JCConf 2020] 用 Kotlin 跨入 Serverless 世代 by Shengyou Fan
[JCConf 2020] 用 Kotlin 跨入 Serverless 世代[JCConf 2020] 用 Kotlin 跨入 Serverless 世代
[JCConf 2020] 用 Kotlin 跨入 Serverless 世代
Shengyou Fan623 views
AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306) by Amazon Web Services
AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306)AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306)
AWS re:Invent 2016: IoT Visualizations and Analytics (IOT306)
Amazon Web Services2.2K views
The fundamentals of AWS Cloud Security 🛠⛅️🚀 by Thanh Nguyen
The fundamentals of AWS Cloud Security 🛠⛅️🚀The fundamentals of AWS Cloud Security 🛠⛅️🚀
The fundamentals of AWS Cloud Security 🛠⛅️🚀
Thanh Nguyen761 views
Riga dev day: Lambda architecture at AWS by Antons Kranga
Riga dev day: Lambda architecture at AWSRiga dev day: Lambda architecture at AWS
Riga dev day: Lambda architecture at AWS
Antons Kranga1.2K views
Amazon Web Services for PHP Developers by Jeremy Lindblom
Amazon Web Services for PHP DevelopersAmazon Web Services for PHP Developers
Amazon Web Services for PHP Developers
Jeremy Lindblom3K views
Managing Your Infrastructure as Code by Travis Williams, Solutions Architect,... by Amazon Web Services
Managing Your Infrastructure as Code by Travis Williams, Solutions Architect,...Managing Your Infrastructure as Code by Travis Williams, Solutions Architect,...
Managing Your Infrastructure as Code by Travis Williams, Solutions Architect,...
Amazon Web Services1.7K views
Automating your Infrastructure Deployment with AWS CloudFormation and AWS Ops... by Amazon Web Services
Automating your Infrastructure Deployment with AWS CloudFormation and AWS Ops...Automating your Infrastructure Deployment with AWS CloudFormation and AWS Ops...
Automating your Infrastructure Deployment with AWS CloudFormation and AWS Ops...
CloudFork by ESUG
CloudForkCloudFork
CloudFork
ESUG629 views
(MBL312) NEW! AWS IoT: Programming a Physical World w/ Shadows & Rules by Amazon Web Services
(MBL312) NEW! AWS IoT: Programming a Physical World w/ Shadows & Rules(MBL312) NEW! AWS IoT: Programming a Physical World w/ Shadows & Rules
(MBL312) NEW! AWS IoT: Programming a Physical World w/ Shadows & Rules
Amazon Web Services10.4K views
(MBL305) You Have Data from the Devices, Now What?: Getting the Value of the IoT by Amazon Web Services
(MBL305) You Have Data from the Devices, Now What?: Getting the Value of the IoT(MBL305) You Have Data from the Devices, Now What?: Getting the Value of the IoT
(MBL305) You Have Data from the Devices, Now What?: Getting the Value of the IoT
Amazon Web Services14.7K views
Java/Scala Lab: Борис Трофимов - Обжигающая Big Data. by GeeksLab Odessa
Java/Scala Lab: Борис Трофимов - Обжигающая Big Data.Java/Scala Lab: Борис Трофимов - Обжигающая Big Data.
Java/Scala Lab: Борис Трофимов - Обжигающая Big Data.
GeeksLab Odessa413 views
Alternator webinar september 2019 by Nadav Har'El
Alternator webinar   september 2019Alternator webinar   september 2019
Alternator webinar september 2019
Nadav Har'El67 views
DevOps Fest 2019. Alex Casalboni. Configuration management and service discov... by DevOps_Fest
DevOps Fest 2019. Alex Casalboni. Configuration management and service discov...DevOps Fest 2019. Alex Casalboni. Configuration management and service discov...
DevOps Fest 2019. Alex Casalboni. Configuration management and service discov...
DevOps_Fest135 views

Recently uploaded

[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo... by
[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...
[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...DataScienceConferenc1
9 views77 slides
Shreyas hospital statistics.pdf by
Shreyas hospital statistics.pdfShreyas hospital statistics.pdf
Shreyas hospital statistics.pdfsamithavinal
5 views9 slides
Inawisdom Quick Sight by
Inawisdom Quick SightInawisdom Quick Sight
Inawisdom Quick SightPhilipBasford
7 views27 slides
Listed Instruments Survey 2022.pptx by
Listed Instruments Survey  2022.pptxListed Instruments Survey  2022.pptx
Listed Instruments Survey 2022.pptxsecretariat4
93 views12 slides
Penetration testing by Burpsuite by
Penetration testing by  BurpsuitePenetration testing by  Burpsuite
Penetration testing by BurpsuiteAyonDebnathCertified
5 views19 slides
4_4_WP_4_06_ND_Model.pptx by
4_4_WP_4_06_ND_Model.pptx4_4_WP_4_06_ND_Model.pptx
4_4_WP_4_06_ND_Model.pptxd6fmc6kwd4
7 views13 slides

Recently uploaded(20)

[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo... by DataScienceConferenc1
[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...
[DSC Europe 23][DigiHealth] Muthu Ramachandran AI and Blockchain Framework fo...
Shreyas hospital statistics.pdf by samithavinal
Shreyas hospital statistics.pdfShreyas hospital statistics.pdf
Shreyas hospital statistics.pdf
samithavinal5 views
Listed Instruments Survey 2022.pptx by secretariat4
Listed Instruments Survey  2022.pptxListed Instruments Survey  2022.pptx
Listed Instruments Survey 2022.pptx
secretariat493 views
4_4_WP_4_06_ND_Model.pptx by d6fmc6kwd4
4_4_WP_4_06_ND_Model.pptx4_4_WP_4_06_ND_Model.pptx
4_4_WP_4_06_ND_Model.pptx
d6fmc6kwd47 views
Ukraine Infographic_22NOV2023_v2.pdf by AnastosiyaGurin
Ukraine Infographic_22NOV2023_v2.pdfUkraine Infographic_22NOV2023_v2.pdf
Ukraine Infographic_22NOV2023_v2.pdf
AnastosiyaGurin1.4K views
Lack of communication among family.pptx by ahmed164023
Lack of communication among family.pptxLack of communication among family.pptx
Lack of communication among family.pptx
ahmed16402314 views
Dr. Ousmane Badiane-2023 ReSAKSS Conference by AKADEMIYA2063
Dr. Ousmane Badiane-2023 ReSAKSS ConferenceDr. Ousmane Badiane-2023 ReSAKSS Conference
Dr. Ousmane Badiane-2023 ReSAKSS Conference
AKADEMIYA20635 views
K-Drama Recommendation Using Python by FridaPutriassa
K-Drama Recommendation Using PythonK-Drama Recommendation Using Python
K-Drama Recommendation Using Python
FridaPutriassa5 views
Best Home Security Systems.pptx by mogalang
Best Home Security Systems.pptxBest Home Security Systems.pptx
Best Home Security Systems.pptx
mogalang9 views
[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init... by DataScienceConferenc1
[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init...[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init...
[DSC Europe 23][Cryptica] Martin_Summer_Digital_central_bank_money_Ideas_init...

Re:Invent 2018 Database Announcements

  • 1. steven@claritycloudworks.com +64 27 620 1237 claritycloudworks.com Steven Ensslen
  • 2. DynamoDB Transactions data = await dynamoDb.transactWriteItems({ TransactItems: [ { Update: { TableName: 'items', Key: { id: { S: itemId } }, ConditionExpression: 'available = :true', UpdateExpression: 'set available = :false, ' + 'ownedBy = :player', ExpressionAttributeValues: { ':true': { BOOL: true }, ':false': { BOOL: false }, ':player': { S: playerId } } } }, { Update: { TableName: 'players', Key: { id: { S: playerId } }, ConditionExpression: 'coins >= :price', UpdateExpression: 'set coins = coins - :price, ' + 'inventory = list_append(inventory, :items)', ExpressionAttributeValues: { ':items': { L: [{ S: itemId }] }, ':price': { N: itemPrice.toString() } } } } ] }).promise(); https://aws.amazon.com/blogs/aws/new-amazon-dynamodb-transactions/ Makes Dynamo “ACID” 
 => suitable for more complex workloads
  • 3. DynamoDB On-Demand https://aws.amazon.com/blogs/database/amazon-dynamodb-on-demand-no- capacity-planning-and-pay-per-request-pricing/ DynamoDB now has two pricing models: 1. Provisioned, which is cheaper if your load is readily forecasted and enough load to cover the cost making the forecast. 
 $0.00074/WCU / 3600 seconds = $0.205 per million writes 2. On-demand, which is more expensive per request, but saves the effort of capacity planning and the cost of excess capacity. $1.423 per million Writes
  • 4. Aurora Global DB https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora- global-database.html A single Aurora database can now be multi-regional. • One region is read/write. • One region is read-only. • Today, failover is entirely your responsibility, including preventing split-brain.
  • 5. RDS on VMWare(preview) https://aws.amazon.com/rds/vmware/ You provide the hardware, AWS provides the software 1. Outsource Database Administration to AWS 2. Burst capacity (scale out) into AWS 3. High Availability and Disaster Recovery via AWS 4. Cloud Migration 
 (alternative to Database Migration Service)
  • 6. Managed Service for Kafka (MSK) (public preview) https://aws.amazon.com/msk/ Feature set very similar to Kinesis - direct competitor MSK use cases: • Existing Kafka application code • Exactly once delivery Pros: • Cheaper if your server is at capacity Cons: • Unit of scaling is more coarse (1 M5.large vs 1 MB/s) • Scaling is not seamless to application.
  • 7. Timestream(preview) https://aws.amazon.com/timestream/ • Timeseries Database • 1000 times faster than RDS for time series data • Time series specific compression and query engine. • Built-in interpolation and smoothing • Serverless
  • 8. Quantum Ledger Database (preview) https://aws.amazon.com/qldb/ • Blockchain as a service • No relation to quantum computing • Immutable and Persistent journal ~= Transaction Log • Reliable and irrefutable “Created By” and “Updated By” • Document data model with SQL • Twice as fast as blockchain => 500 times slower than RDS? • AWS has been using this internally for billing for “years” • Finally a means to implement the intent of the US Sarbanes-Oxley Act of 2002
  • 9. Lake formation (preview) https://aws.amazon.com/lake-formation/ • No cost, just co-ordinates other services • mostly generates AWS Glue (pyspark) modules • Undifferentiated heavy data lake lifting: • Ingest • Tranform to ORC or Parquet • Encrypt • Secure
  • 10. Clarity Cloudworks illuminating issues before they become problems