Download to read offline
Uploaded byScyllaDB
DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
The document presents a technical comparison between DynamoDB and ScyllaDB, highlighting ScyllaDB's advantages in terms of cost, latency, and performance scalability. It outlines migration paths, tools, and best practices for transitioning from DynamoDB to ScyllaDB, noting significant cost reductions and improved operational efficiency. Case studies, including Disney Streaming's migration, illustrate successful implementations and provide insights on technology integration and performance monitoring.
More Related Content
Similar to DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
DynamoDB to ScyllaDB: Technical Comparison and the Path to Success
- 1. DynamoDB to ScyllaDB Atechnical comparison and path to success Tim Koopmans, Product Experience Guilherme Nogueira, Solutions Architect
- 2. ■ Introduction ■ UnderstandingScyllaDB ■ Comparing DynamoDB and ScyllaDB ■ Migration Options and Tools ■ Case Studies and Best Practices ■ Conclusion Presentation Agenda
- 3.
- 4. Why choose ScyllaDB? BestHigh Availability in the industry Best Disaster Recovery in the industry Best scalability in the industry Best Price/Performance in the industry Auto-tune - out of the box performance Compatible with Cassandra & DynamoDB No Lock-in Open Source Software
- 5. Why do DynamoDBcustomers switch?
- 6. ■ Adequate provisionedcapacity but high throttling ■ Throttling despite activated AWS Application Auto Scaling ■ Throttling in on-demand capacity mode ■ Presence of a hot partition ■ Traffic exceeding account throughput quotas ProvisionedThroughputExceededException
- 7.
- 8. The benchmark Provisioned Throughput 200KRCU 250K WCU Cluster Size i4i.2XL x 3 YCSB Loaders c5.2XL x 8 18-150K ops/sec lp.scylladb.com/price-performance-dynamoDB-benchmark-offer
- 9.
- 10.
- 11.
- 12. Throughput ■ DynamoDB ■ Throughputlimited at partition level ■ ScyllaDB ■ Balanced / prioritized workloads across the entire cluster
- 13. Cost DynamoDB ScyllaDB Provisioned cost$30k/mo $4.5k/mo 7X lower On-demand cost $206k/mo $4.5k/mo 45X lower P99 latency 20.2 ms 11.1 ms 2X faster
- 14. Migrating from DynamoDB toScyllaDB
- 15. ■ ScyllaDB isclosely related to DynamoDB Similarities Dynamo Paper (Amazon) BigTable Paper (Google) 2006 2007 2008 2015 2012
- 16. ■ ScyllaDB implementsCQL ■ Query language ■ Protocol ■ Data model ■ Alternator brought in DynamoDB API compatibility ■ Transports are popular standards ■ HTTP and JSON Comparison
- 17. Comparison Aspect CQL DynamoDBAPI Transport CQL binary protocol HTTP and JSON Protocol Cluster-aware Single endpoint Query language Prepared statements No prepared statements Write model Conflict-free Replicated Data Types (CRDT) / Lightweight Transactions (LWT) Read-Modify-Write
- 18. Comparison - Schema ■ScyllaDB enforces a schema ■ Partition Key(s), Clustering Key(s) ■ CQL types ■ UDTs ■ Materialized Views, Local Secondary Indexes, Global Secondary Indexes ■ CDC ■ DynamoDB enforces Keys ■ Primary Key, Sorting Key ■ DynamoDB Types ■ Local Secondary Indexes, Global Secondary Indexes ■ DynamoDB Streams
- 19. Depending on thechosen path ■ DynamoDB to ScyllaDB Alternator ■ No application rewrite ■ Table should work as-is ■ Run DDB-compatible workloads anywhere! ■ DynamoDB to CQL ■ Application and Table changes required ■ Feature-rich, cluster aware ■ High performance Migration Paths
- 20. ■ DynamoDB toScyllaDB Alternator ■ Use DynamoDB-API compatible libraries and tools ■ Dual writes ■ Change application to target both DDB and Alternator ■ Leverage DynamoDB Streams + Lambda ■ Historical data lift-and-shift ■ Read from DDB table ■ Consumes RCUs ■ Beware of throttling and $ ■ Export to S3 Migration Paths - DDB to Alternator
- 21. ■ DynamoDB toScyllaDB using CQL ■ Use DynamoDB-API compatible tools ■ Python boto3 and Java AWS SDK ■ Native CQL libraries for interacting with ScyllaDB ■ Dual writes ■ Change application to target both DDB and ScyllaDB ■ Historical data lift-and-shift ■ ETL from DDB types to CQL types ■ Read from table or S3 Migration Paths - DynamoDB to CQL
- 22. Libraries designed towork with DynamoDB API works with Alternator ■ Python boto3, Java AWS SDK… ■ Kafka Streams Connector ■ NoSQL Workbench ■ Spark Tooling
- 23. Tooling - CQL ■Built-in shard awareness on ScyllaDB drivers ■ Enabled by default ■ Supports many languages: ■ Also present in connectors:
- 24. ■ Reading tablescan trip throttling ■ Impact to Production apps ■ Reading JSON from S3 has its challenges ■ Not everything abides to JSON standards ■ Better to stick with official libraries to import data ■ Schema changes over the years Challenges
- 25. Case Studies AndBest Practices
- 26. ■ Cost ■ LimitedSDK ■ Write throughput ■ Why ScyllaDB? ■ Multi-region ■ ICS, CDC, Seastar shared-nothing, drivers ■ 1ms writes, single-digit read p99 Why did Disney Streaming move out of DynamoDB?
- 27. Real World Examples DigitalTurbine migrated from DynamoDB to ScyllaDB Alternator on GCP ■ Ease of migration ■ No application refactor ■ Reduce cost (20% cost reduction upfront, with opportunity for more) ■ Better performance ■ ScyllaDB Alternator cluster, Cluster ~2,000 ops/sec (90% reads , 10% Writes) ■ Leveraging DynamoDB API ■ DynamoDB ~1,400 ops/sec (90% reads , 10% Writes) ■ Observed light throttling on the DynamoDB side
- 28. ■ Keep vitalmetrics close ■ Use Scylla Monitoring stack ■ Know your data and your application ■ Learn about historical changes to your application ■ Keep it simple Best Practices
- 29. ■ Know thetechnology ■ Use the best tool for the job ■ Apples to Apples migration might be easier ■ Migrating to a different protocol might open many possibilities ■ PoC the technology ■ Application integration, smoke tests ■ Performance comparison ■ Cost Do’s and Don’ts
- 30. ■ Use thebest tool for the job ■ When interacting with DDB use default libraries ■ CQL Shard-aware drivers ■ Full compatibility is impossible ■ Direct S3 export ■ CloudFormation integration for creating tables Do’s and Don’ts - Cont'd
- 31.
- 32. Conclusion - ScyllaDB Universityis 100% FREE - Lessons on the DynamoDB API - Migration - Many useful blogs - Customer stories - Migrating from DynamoDB to ScyllaDB can be extremely beneficial - Easy to get started - Also available as a Managed Service
- 33.